Below is an exerpt of the remarks of Michael Campbell (Vice President Product Management, Pro/ENGINEER) on the Wildfire OCUS Benchmark v3.2

I agree with him on these issues and we are currently working together to make the result more accurate.

1. Extra Features in the Pro/E Wildfire Version of the test
   We analyzed the test that you have posted on your website, and we found that in the Pro/E Wildfire version of the test, you created an extra pattern feature, which resulted in the creation of 100 extra datums. Then, the part which contains the 100 extra datum features, is then assembled 100 times in an assembly, which is used to measure some of the performance actions. With 10,000 extra features, we believe that this definitely has a negative impact on some specific aspects of the benchmark ("11 Model tree expansions" and "Create Drawing" tests ), and could slow other actions as well.
   Obviously this oversight is detrimental to the premise that the benchmark compares the same actions in both Pro/E 2001 and Pro/E Wildfire.
   
2. Use of commands from older versions of Pro/ENGINEER
   We have also noted that the test run on Pro/E Wildfire contains many command sequences which were originally created in Pro/ENGINEER 2001. There are two negative implications of doing this.
   
   • In certain portions of the test, old, Pro/ENGINEER 2001 Menu Manager picks are the instructions in the test. Since the User Interface has changed so much in Pro/E Wildfire, the presence of these menu commands means that during the execution of the benchmark, Pro/ENGINEER is spending time interpreting these “old-style” menu picks into Pro/E Wildfire User Interface commands, and then executing these commands. This artificial translation is partially responsible for the slower results. Some examples of these “old-style” menu picks are "Input #PART NAME in #SEL BY MENU", "#SHOW", which appear in the sections testing "10 Explode", and Spin/Pan/Zoom.
   • Furthermore, it is a known fact that the Wildfire embedded browser can cause existing (Pre-Wildfire) trail files to run slower. To address that, code was built into Pro/ENGINEER Wildfire to offset this slowdown. That is, if you perform a simple test in Pro/E Wildfire of opening a part, and generating a BOM report for it, using both a trail file created in Pro/E 2001, and then a trail file created in Pro/E Wildfire, the test will complete much faster with the trail file which was originally created in Pro/E Wildfire. This is due to the fact that trail files created in Pro/ENGINEER Wildfire contain “embedded browser timer lines”. These lines enable graphics optimization and can provide several hundred percent improvements when running trail files which test drawings, as well as other capabilities as well.

The point I’m trying to make, and I hope you would agree, is that if the intent is to measure performance, then the benchmark should measure the time to complete the task, rather than the time to run a trail file. In order to fairly measure performance, it is important to run a Pro/E Wildifre trail file in Pro/E Wildfire.

These issues lead to some of the differences between our results and the results you published. For example:

• X-Section shows (GR) - PTC trail (native to Pro/E Wildfire) does not show degradation as opposed to your reported 285% performance drop. It could possibly be related to trail playback differences.
• Create Drawing (CP) - PTC shows marginal slowdown, while you report 165% degradation. We believe the reason is the extra datum features, and have confirmed this by re-running the test.
• Postscript file creation (CP+DI) - This is a known Issue, and there is an SPR filed. PTC test shows a 20% slowdown while your test shows 210%. The possible reason is due to the additional datum features introduced by the pattern.

There are other reported slow downs in your analysis that we expected, due to functional changes in Pro/ENGINEER Wildfire. For example, you reported that it took about four times as long to create a DXF file. This is due to the fact, that DXF file quality was improved in a more recent build of Pro/ENGINEER 2001, as well as in Pro/E Wildfire. The exported entities are now output as SPLINES, not POLYLINES, resulting in higher fidelity exported data. The old behavior still exists, and can be seen by setting the config.pro option: intf2d_out_enhanced_ents to none/hatch_only. Doing so will result in performance (and data quality) comparable with what you saw in Pro/ENGINEER Release 2001 when you ran your test originally.

With these points in mind, I would like to share with you our results from running your benchmark internally on some of the latest builds of Pro/ENGINEER 2001 and Pro/ENGINEER Wildfire. In summary, overall we found less than an 8% decrease in performance. While some there is obviously room for improvement, I assure you we are continually working on improving the performance of subsequent builds of Pro/E Wildfire. In fact, we have recently increased by 25% the resources dedicated to maintaining Pro/E 2001 and Pro/E Wildfire, from a quality and performance perspective.

Furthermore, we have made many significant performance improvements in Pro/ENGINEER Wildfire, which are not reflected in the OCUS Benchmark. For a sampling, please review the Pro/E Wildfire Performance Technology Highlight, located here. (http://www.ptc.com/WCMS/files/7403en_file1.pdf)

I have included our perspective on some of the other degradations you have identified.

• Hidden line redraws with FastHLR (GR) - Known issue: SPR is filed. PTC notes a 65% degradation compared to Olaf's 100%, but the difference varies based on the platform used and is 100% on certain other platforms inside PTC.
• Shaded mouse spins (GR) - Known issue: SPR is filed. Approximately 50% degradation noted, but an accurate comparison between Pro/E 2001 and Pro/E Wildfire is practically impossible since the trail has to be re-created and the exact motions cannot be exactly reproduced by hand.
• Shaded redraws (GR) - Known issue: SPR is filed. PTC notes a 50% degradation compared to Olaf's 100%, possibly due to differing builds of 2001 and Wildfire used.
• Wireframe mouse zooms (GR) - Known issue: SPR is filed. Approximately 50% degradation noted by PTC as opposed to Olaf's 85%, but an accurate comparison between Pro/E 2001 and Pro/E Wildfire is practically impossible since the trail has to be re-created and the exact motions cannot be exactly reproduced by hand.
• Explode translations (GR) - Known issue: SPR is being fixed. Approximately 60% degradation, but an accurate comparison between Pro/E 2001 and Pro/E Wildfire is practically impossible since the trail has to be re-created and the exact motions cannot be exactly reproduced by hand. Selection in Olaf's trail input is also different.
• Show Dimensions (GR) - Known Issue: PTC shows a 40% slowdown as opposed to Olaf's 65%. This degradation was agreed to be acceptable by Product Management and Development. The difference is possibly attributed to trail playback differences.