You are on page 1of 8

Testing: Revit Architecture 2008 and 2009 with Riverbed Steelhead Appliance and Steelhead Mobile

Executive Summary
Revit® Architecture from Autodesk is building design software which enables architects to visualize ideas on the fly with 3D views and fully rendered scenes. More information on Revit Architecture can be found at the following URL:
http://usa.autodesk.com/adsk/servlet/index?siteID=123112&id=8479253

Application performance testing of Revit Architecture using Steelhead and Steelhead mobile shows significant improvement for both the cold and warm pass.

Revit 2008 Results Summary
Total Data Transferred 396.2 Megabytes 76 Megabytes 6 Megabytes 166 8 Megabytes Megabytes Total Time to Complete 68.0 minutes 20.0 minutes 14.6 minutes 28.9 14.6 minutes minutes Data Reduction 80.8% 98.5% 58.1% 98.0% Time Improvement 3.4 4.7 2.4 4.7 x x x x

Baseline Steelhead Cold Steelhead Warm SHMobile Cold SHMobile Warm

Note: the SHMobile data transferred is the sum of the both clients (individual data stores vs. single data store in the SH appliance).

Revit 2009 Results Summary
Total Data Transferred 200 Megabytes 20 Megabytes 4 Megabytes Total Time to Complete 23.5 minutes 5.7 minutes 3.9 minutes Data Reduction 90% 98% Time Improvement 4.2 6.0 x x

Baseline Steelhead Cold Steelhead Warm

Performance Testing Overview:
The purpose of this test is to determine the benefits of the Steelhead appliance and Steelhead Mobile software on Revit Architecture 2008 software in a WAN environment. The testing simulates two Revit users working on separate local workstations. Each workstation has the Revit Architecture 2008 software installed. Each user opens a common Central File on a server located across a WAN, saves a Local copy to their computer, performs some work (which involves getting permissions and borrowing elements from the Central File), and then publishes their changes to the Central File. The tests use a custom API tool

1

(WorkScheduler.exe) so that the Saves to Central are near-simultaneous, simulating typical real-life Central File conflicts The simulation of the 2 users is done using a script provided by Autodesk. The script directs each of the 2 users perform the following six operations. Operations 1 and 5 do not access files across a network, resulting in consistent times for all the tested topologies. 1. 2. 3. 4. 5. 6. Save as Local_File_User.rvt Make some changes to User local file User Save to Central User Reload Latest Make some changes to User Local File User Save to Central

The WAN environment used in this case is a simulated T1 with 100 ms of latency. A baseline test was run without any Steelhead appliances, and then with the Steelhead appliances and finally with the Steelhead mobile client software. The server and workstations are running Windows XP with service pack 2 as the OS. The client workstations are running the June 2007 release of Revit Architecture 2008. Steelheads 300s with version 4.1 software are used for Steelhead Mobile testing version 1.1 of the software was used.

Topology 1: Revit Architecture 2008 without the Steelheads.
The Revit operations would not complete without adjusting the synch timer of the software application in this topology. The default value of the timer is 600 seconds. It was adjusted to 1200 seconds.

2

The following is a graph of the baseline performance of the Revit Architecture software across a WAN. The graph combines the 6 operations for each of the 2 users. For individual breakdown of test results refer to Appendix A.

W AN Utilization (bi-directional) 250 200 150 100 50 0 1 331 661 991 1321 1651 1981 2311 2641 2971 3301 3631 3961 Time (seconds) Baseline

___ Baseline 396.2 Megabytes

Throughput (kbyte)

68 minutes

Topology 2: Revit Architecture 2008 with Steelheads Appliances
In the second topology the same tests were performed as in topology 1 with the addition of the Riverbed Steelhead appliances. The test was performed with both cold and warm data store on the Steelhead. The following topology was used for the testing. The following graph shows the comparison of baseline performance of the Steelheads with cold and warm data store passes. This is the combined stats of the 6 different functions that were run in the test script. For detailed breakdown of each individual test result refer to Appendix A.

3

WAN Utilization (bi-directional)
250

200

Throughput (kbyte)

150

100

50

0 1 208 415 622 829 1036 1243 1450 1657 1864 2071 2278 2485 2692 2899 3106 3313 3520 3727 3934 Time (seconds) Baseline Cold Warm

___ Baseline ___ Cold ___ Warm

396.2 Megabytes 76 Megabytes 6 megabytes

68 minutes 20 minutes 14 minutes 35 seconds

Topology 3: Revit Architecture 2008 with Steelhead Mobile
The third topology performed the same tests using Steelhead Mobile software on the clients and the Steelhead appliance at the server site. The tests were performed with both cold and warm data store on the Steelhead. The following topology was used for the testing.

4

The following is a graph comparing the baseline, cold and warm performance of the Steelhead Mobile software. Results of SH Mobile and SH appliance was similar for warm transfer. In the cold performance the topology using just Steelhead appliances performed better. This is expected due to the shared datastore on the client side when the Steelhead appliance was used, vs. individual data stores for the Steelhead Mobile client. For detailed breakdown of test results refer to Appendix A.

WAN Utilization (bi-directional)
300 250 Throughput (kbyte) 200 150 100 50 0 1 331 661 991 1321 1651 1981 2311 2641 2971 3301 3631 3961 Time (seconds) Baseline Cold Warm

___ Baseline ___ Cold ___ Warm

396.2 Megabytes 166 Megabytes 8 megabytes

68 minutes 28 minutes 51 seconds 14 minutes 36 seconds

5

Topology 4: Revit Architecture 2009 with Steelhead Appliances
Testing was also performed with Revit 2009 and Steelhead appliances to ensure continued interoperability between the 2 products. It was also mentioned by several Revit customers that the average size of a shared Revit file is 150 MB so testing was modified to use a 200 MB file to simulate a more real life environment. Testing was performed with a simulated 5mb link with 100 ms of delay. The testing with Revit 2009 was streamlined to test only the operations that are WAN intensive, namely, Save_to_Central and Reload_test.

“ User1___ Baseline User1___ Cold User1___ Warm User2___ Baseline User2___ Cold User2___ Warm 1412 seconds 340 seconds 234 seconds 1411 seconds 339 seconds 223 seconds

90% data reduction 98% data reduction

90% data reduction 98% data reduction

6

Appendix A – Detailed performance results Revit Architecture Performance Tests
Baseline - No Steelheads
Operation User 1 Save as Local_File_User1.rvt Make some changes to User1 User1 Save to Central User1 Reload Latest Make some changes to User1 Local File User1 Save to Central Operation User 2 Save as Local_File_User2.rvt Make some changes to User2 User2 Save to Central User2 Reload Latest Make some changes to User2 Local File User2 Save to Central Run 1 19 seconds 712 seconds 859 seconds 223 seconds 63 seconds 616 seconds Run 1 14 seconds 67 seconds 860 seconds 224 seconds 62 seconds 617 seconds Run 2 19 seconds 710 seconds 797 seconds 223 seconds 62 seconds 604 seconds Run 2 14 seconds 77 seconds 795 seconds 222 seconds 61 seconds 603 seconds Run 3 19 seconds 710 seconds 823 seconds 223 seconds 62 seconds 608 seconds Run 3 14 seconds 67 seconds 860 seconds 222 seconds 61 seconds 617 seconds Avg 19 711 826 223 62 609 Avg 14 73 838 223 61 612

Steelheads Cold Transfer
Operation User 1 Save as Local_File_User1.rvt Make some changes to User1 User1 Save to Central User1 Reload Latest Make some changes to User1 Local File User1 Save to Central Operation User 2 Save as Local_File_User2.rvt Make some changes to User2 User2 Save to Central User2 Reload Latest Make some changes to User2 Local File User2 Save to Central Run 1 19 seconds 149 seconds 269 seconds 26 seconds 56 seconds 299 seconds Run 1 17 seconds 66 seconds 267 seconds 27 seconds 55 seconds 301 seconds Run 1 19 seconds 147 seconds 163 seconds 27 seconds 56 seconds 122 seconds Run 1 14 seconds 66 seconds 186 seconds 26 seconds 55 seconds 123 seconds Run 2 18 seconds 144 seconds 289 seconds 27 seconds 56 seconds 252 seconds Run 2 14 seconds 65 seconds 288 seconds 27 seconds 55 seconds 254 seconds Run 2 19 seconds 147 seconds 185 seconds 27 seconds 56 seconds 122 seconds Run 2 14 seconds 65 seconds 163seconds 27 seconds 55 seconds 122 seconds Run 3 19 seconds 146 seconds 267 seconds 27 seconds 56 seconds 173 seconds Run 3 14 seconds 66 seconds 266 seconds 28 seconds 55 seconds 174 seconds Run 3 19 seconds 146 seconds 166 seconds 27 seconds 56 seconds 122 seconds Run 3 14 seconds 66 seconds 164 seconds 28 seconds 55 seconds 123 seconds Avg 18 146 275 27 56 241 Avg 15 66 273 27 55 243 Avg 19 147 171 27 56 122 Avg 14 66 171 27 55 123

Steelheads Warm Transfer
Operation User 1 Save as Local_File_User1.rvt Make some changes to User1 User1 Save to Central User1 Reload Latest Make some changes to User1 Local File User1 Save to Central Operation User 2 Save as Local_File_User2.rvt Make some changes to User2 User2 Save to Central User2 Reload Latest Make some changes to User2 Local File User2 Save to Central

7

Steelhead Mobile Cold Cold Transfer
Operation User 1 Save as Local_File_User2.rvt Make some changes to User2 User2 Save to Central User2 Reload Latest Make some changes to User2 Local File User2 Save to Central Operation User 2 Save as Local_File_User2.rvt Make some changes to User2 User2 Save to Central User2 Reload Latest Make some changes to User2 Local File User2 Save to Central Run 1 22 seconds 402 seconds 322 seconds 113 seconds 56 seconds 325 seconds Run 1 13 seconds 67 seconds 322 seconds 112 seconds 56 seconds 326 seconds Run 2 19 seconds 390 seconds 323 seconds 112 seconds 55 seconds 320 seconds Run 2 12 seconds 70 seconds 303 seconds 110 seconds 55 seconds 323 seconds Run 3 19 seconds 398 seconds 340 seconds 111 seconds 62 seconds 325 seconds Run 3 12 seconds 70 seconds 320 seconds 110 seconds 55 seconds 323 seconds Avg 20 397 328 112 58 323 Avg 12 69 315 111 55 324

Steelheads Mobile Warm Transfer
Operation User 1 Save as Local_File_User1.rvt Make some changes to User1 User1 Save to Central User1 Reload Latest Make some changes to User1 Local File User1 Save to Central Operation User 2 Save as Local_File_User2.rvt Make some changes to User2 User2 Save to Central User2 Reload Latest Make some changes to User2 Local File User2 Save to Central

Run 1 22 seconds 218 seconds 162 seconds 26 seconds 59 seconds 125 seconds Run 1 13 seconds 66 seconds 161 seconds 27 seconds 58 seconds 125 seconds Run 1 20 seconds 215 seconds 283 seconds 12 seconds 33 seconds 101 seconds Run 1 16 seconds 144 seconds 283 seconds 12 seconds 32 seconds 101 seconds

Run 2 20 seconds 238 seconds 178 seconds 26 seconds 56 seconds 122 seconds Run 2 14 seconds 65 seconds 178 seconds 27 seconds 55 seconds 123 seconds Run 2 20 seconds 211 seconds 275 seconds 13 seconds 31 seconds 98 seconds Run 2 19 seconds 138 seconds 273 seconds 13 seconds 31 seconds 98 seconds

Run 3 22 seconds 262 seconds 160 seconds 28 seconds 58 seconds 125 seconds Run 3 14 seconds 66 seconds 160 seconds 28 seconds 58 seconds 126 seconds Run 3 16 seconds 218 seconds 276 seconds 12 seconds 31 seconds 98 seconds Run 3 18 seconds 140 seconds 276 seconds 13 seconds 31 seconds 97 seconds

Avg 21 239 167 27 57 124 Avg 13 66 166 27 57 125 Avg 19 214 278 12 32 99 Avg 17 139 277 12 31 98

LAN Transfer
Operation User 1 Save as Local_File_User1.rvt Make some changes to User1 User1 Save to Central User1 Reload Latest Make some changes to User1 Local File User1 Save to Central Operation User 2 Save as Local_File_User2.rvt Make some changes to User2 User2 Save to Central User2 Reload Latest Make some changes to User2 Local File User2 Save to Central

8