Overview

The ACS Performance framework was created primarly as a means to determine the performance limitations of various ACS APIs. This includes very low- level tests such as determining how many method invocations can be invoked on a component per second to more abstract tests such as determining how long it takes to start the core of ACS. The framework has been created in such a way that not only is it useful to ACS, it can be used by other ALMA software subsystems as well. It consists of:

• Profiler objects in C++, Java, and Python. These profilers are essentially stopwatches that obtain interesting data about a particular block of code.
• A bash script named acsutilProfiler which profiles entire executables.
• A report generator which turns the raw output from the profilers into human-readable reports.

Interesting ACS 4.1.0 Performance Facts

On a single host:

• The amount of time it takes to compile Java files has decreased by nearly one-half since ACS 4.0
• The acsStartORBSRVC command executes nearly 10% slower under ACS 4.1
• The fluctuation of invocation times of Java methods varies greatly with 4.1 compared to ACS 4.0.
• C++ logging performance is by and far at the same level it was with ACS 4.0. This does not hold true for large log sizes though (bigger than 1 Kilobyte) where the amount of time it takes to send a log can be roughly ten times what it used to be. Java logging has taken a small performance hit as well while Python logging has improved overall.
• The fluctuation of invocation times of sending Python events is quite large compared to 4.0, but overall the numbers remain about the same. There is little change with the Java and C++ APIs.
• For ACS exceptions of significant depth (200 or more), performance has significantly increased by what is generally a factor of ~2.
• There seems to be a slight improvement with the Bulk Data API under most cases.

• The data retrieved from the remote tests is essentially in agreement with the local tests. The only change here is that performance losses/gains are not as noticable because of the added overhead of network communications.
• Bulk Data performance tests hang. To be investigated.

High-level Guide to Using the ACS Benchmarking Suite in Your Own Code

To test the performance of your own code, the following simple steps should be performed:

1. Select a block of code to be analyzed.
2. Write a client which invokes the chosen method n times. N should be a reasonably large number to obtain accurate measurements.
3. Incorporate Profiler objects into your client.
4. Startup ACS and any applicable containers.
5. Run the client and save it's output to file.
6. Import the file containing the client's output into a database using tools provided by ACS.
7. Generate HTML reports using the performance database.

Old Performance Facts

Interesting ACS 4.0.1 Performance Facts

On a 1 Gig Ethernet connection utilizing remote containers:

• It takes half a millisecond to invoke a C++ or Python component's method and 2.5ms to invoke a Java component's method.
• It takes 46ms to invoke a C++ component's method which returns 500KBytes of data from C++ or Python clients. From Java, it takes 87ms.
• Python and Java perform nearly one order of magnitude slower than C++ with respect to logging.
• Java clients consistently catch CORBA exceptions faster than C++/Python (by roughly 15%).
• Using the Bulk Data API, data rates of 16MB/second are achieved for large data sets. CPU speed played a large role in this.
• The Event Channel API has no problems handling one-thousand events per second of size 1KB in a very simple scenario.
• In general, C++/Python are neck and neck with respect to performance although Java is fine for everything except large data sets.

	Name	Last modified	Size	Description
	Parent Directory		-
	ComponentBulkDataPerf.html	2005-06-06 02:34	75K	Performance of the Bul>
	ComponentErrorPerf.html	2006-02-04 00:39	91K
	ComponentEventPerf.html	2006-02-04 00:39	37K
	ComponentLoggingPerf.html	2006-02-04 00:39	40K
	ComponentMethodPerf.html	2006-02-04 00:39	136K
	MakefilePerf.html	2006-02-04 00:48	12K
	RemoteComponentBulkDataPerf.html	2004-11-29 20:11	48K	Performance of the Bul>
	RemoteComponentErrorPerf.html	2006-02-05 20:33	91K
	RemoteComponentEventPerf.html	2006-02-05 20:33	37K
	RemoteComponentLoggingPerf.html	2006-02-05 20:33	40K
	RemoteComponentMethodPerf.html	2006-02-05 20:33	135K
	StartupPerf.html	2006-02-04 18:46	17K

Overview

The ACS Performance framework was created primarly as a means to determine the performance limitations of various ACS APIs. This includes very low- level tests such as determining how many method invocations can be invoked on a component per second to more abstract tests such as determining how long it takes to start the core of ACS. The framework has been created in such a way that not only is it useful to ACS, it can be used by other ALMA software subsystems as well. It consists of:

• Profiler objects in C++, Java, and Python. These profilers are essentially stopwatches that obtain interesting data about a particular block of code.
• A bash script named acsutilProfiler which profiles entire executables.
• A report generator which turns the raw output from the profilers into human-readable reports.

Interesting ACS 4.1.0 Performance Facts

On a single host:

• The amount of time it takes to compile Java files has decreased by nearly one-half since ACS 4.0
• The acsStartORBSRVC command executes nearly 10% slower under ACS 4.1
• The fluctuation of invocation times of Java methods varies greatly with 4.1 compared to ACS 4.0.
• C++ logging performance is by and far at the same level it was with ACS 4.0. This does not hold true for large log sizes though (bigger than 1 Kilobyte) where the amount of time it takes to send a log can be roughly ten times what it used to be. Java logging has taken a small performance hit as well while Python logging has improved overall.
• The fluctuation of invocation times of sending Python events is quite large compared to 4.0, but overall the numbers remain about the same. There is little change with the Java and C++ APIs.
• For ACS exceptions of significant depth (200 or more), performance has significantly increased by what is generally a factor of ~2.
• There seems to be a slight improvement with the Bulk Data API under most cases.

• The data retrieved from the remote tests is essentially in agreement with the local tests. The only change here is that performance losses/gains are not as noticable because of the added overhead of network communications.
• Bulk Data performance tests hang. To be investigated.

High-level Guide to Using the ACS Benchmarking Suite in Your Own Code

To test the performance of your own code, the following simple steps should be performed:

1. Select a block of code to be analyzed.
2. Write a client which invokes the chosen method n times. N should be a reasonably large number to obtain accurate measurements.
3. Incorporate Profiler objects into your client.
4. Startup ACS and any applicable containers.
5. Run the client and save it's output to file.
6. Import the file containing the client's output into a database using tools provided by ACS.
7. Generate HTML reports using the performance database.

Old Performance Facts

Interesting ACS 4.0.1 Performance Facts

On a 1 Gig Ethernet connection utilizing remote containers:

• It takes half a millisecond to invoke a C++ or Python component's method and 2.5ms to invoke a Java component's method.
• It takes 46ms to invoke a C++ component's method which returns 500KBytes of data from C++ or Python clients. From Java, it takes 87ms.
• Python and Java perform nearly one order of magnitude slower than C++ with respect to logging.
• Java clients consistently catch CORBA exceptions faster than C++/Python (by roughly 15%).
• Using the Bulk Data API, data rates of 16MB/second are achieved for large data sets. CPU speed played a large role in this.
• The Event Channel API has no problems handling one-thousand events per second of size 1KB in a very simple scenario.
• In general, C++/Python are neck and neck with respect to performance although Java is fine for everything except large data sets.