This action might not be possible to undo. Are you sure you want to continue?
Tools and techniques
Mike Godfrey Information Technology Services / The University of Texas at Austin
Look at three java projects • Different architectures adopted • Problems encountered and lessons learnt • Tools and other useful techniques
Who are we?
• Remains of centralized data processing • Part of the unit providing IT services across campus • Traditionally mainframe based applications with web front-ends scripted in a simple in-house language • Changing mission as our administrative application development role is done by departments • Project list that requires a much more powerful language pushing adoption of Java • Changes are forcing developer to acquire new skills and ways of working – java camp.
The Systems • Clips – a web-based file system for distributing course materials • Jwebagent – a java runtime for our in-house scripting language • Tech lounge – a web-based developer documentation store .
Clips – file system .
intuitive interface Handle all file types Accessible from any location Date controlled display of materials Easy transfer of materials from class to class Work well with existing mainframe class management system • Work within the UTDirect portal framework .Clips Requirements • • • • • • A simple to use.
Basic Design Choices • File system metaphor for user interface • Browser access for both students and faculty • Use native XML database to model the file system in a recursive schema • Store binary files in same XML database • JSP with scriptlets for display • clips interface .
directorynodes and binary-nodes • One XML document contains all meta information needed to build file structure for a given class • Easy to re-build hierarchy of resources from XML tree • Database uses schema to specify stored document structure .Use of XML for Meta Data • File system metaphor seems ideally suited to the tree structure of XML • Meta data structure based on two nodes types.
etc • Binary nodes store information about a binary file: name. content type. foreign key. description. etc • Directory nodes can contain either binary nodes or other directory nodes (hierarchy) • clips meta data .Meta-Data Details • Directory nodes store information about a directory: name. size. creation time.
Clips Architecture XML database (binaries and Meta-data) Tamino api Broker api JSP presentation layer Authorization and class info Helper Classes User mainframe Adabas Broker Natural Existing class management system .
New Issues for us • Handling and auditing file uploads from web pages (multipart encoded forms) • Building and manipulating XML records • How to best make large scale use of JSPs in a complex Java system • How to simplify communication with mainframe .
jar – used to parse the multipart encoded forms • Entire-X broker ACI for Java • Java stub generator for mainframe modules • Tamino .Useful Tools • Log4j – used for debugging and recording production exceptions (system written before Java 1.native XML database .4) • JDOM – very user friendly api for manipulating the XML • cos.
Lessons Learnt • Be careful of scriplets in JSP. they expand to fill the available space! • Better to use a controller servlet to organize navigation through JSPs • XP and pair-programming works • A Java developer’s best friend is Goggle .
Alternative JSP architecture View jsp Maintain Update Copy jsp jsp jsp View jsp Maintain jsp Update jsp Copy jsp Controller servlet Auth Task Form Resource Parser List Copy Task Update Task Auth Task Form Resource Parser List Copy Task Update Task mainframe XML store mainframe XML store .
Jwebagent .WBX file Jwebagent Jwebagent HTML file .
rebuilds the C++ virtual machine and executes it. • Pro: easy and productive for creating web front ends to mainframe applications.WBX file • Runtime reads .What is Webagent? • A scripting language with a syntax like Natural (mainframe language) • Scripts transformed into C++ objects forming a virtual machine which is serialized to a .WBX file. • Con: limited functionality .
Why Jwebagent? • Same as Webagent but the rebuilding of the runtime is done in Java …plus • Enable java classes to run existing Webagent scripts • Enable Webagent scripts to use java classes • Improve execution speed – no process forking and script buffering .
literals . and handling errors for an individual file request • File loader – re-builds the serialized script objects • Context objects – handle data specific to that particular request • VmOpCodes – functional elements of the VM • Data objects – variables.Main Runtime Elements • Servlet – handles all execution requests for a WBX file • Executor – coordinates loading file. running the VM.
Runtime broker broker broker file request vmOp vmOp vmOp vmOp vmOp vmOp Serial code object Servlet Executor Builds then executes Script context http req env http resp Data buffer Request context WBX file .
org) JProbe – application analyzer (Quest Software) JMeter – load testing suite (Apache. WinCVS.org) HttpExplorer – gui http request and response tool (in-house) Jedit – text editor (Jedit.org) IntelliJ IDEA – Java IDE (Jet Brains) CVS (Tortoise.Useful Tools • • • • • • • • Junit – unit testing framework (junit. Putty for SSH) Xvi32 – hex editor .
Junit – unit testing • Provides easy way to set and teardown test situations • Automated running of many tests • Organize tests in suites and suites into metasuites • Text or gui UI for test reporting • Refactor with confidence! (very valuable) • Trick is to know what tests to write • Good with Jwebagent as it is not database driven .
JProbe – code analyser • Threadalizer – finds deadlocks and conflicts (did not show any problems for Jwebagent) • Memory Debugger – shows memory use by classes and can find hanging objects (useful – strings to byte arrays) • Profiler – tracks objects and the call pattern. records method time and number of calls (most useful tool –identify prime objects to optimize) .
JProbe Code Optimizations • Strings to byte arrays – saved memory on string generation • io to nio gave 25% increase in through-put • In-lining of frequently used methods to reduce calling overhead • Combining of related classes when frequently constructed • Un-doing OOP improved performance! .
files. JDBC. tables.JMeter – load testing • Build and store test plans • Control number of threads. etc • Build tests with logical flows to suit application • Apply pre and post requests during tests • Very valuable for performance tuning (with JProbe) . SOAP and web services. ramp-up times and thread delay to simulate different testing scenarios • Test http apps. LDAP. etc • Display test results in graphs.
JVM Tuning • Changed default of client mode to server mode (20% improvement) • Other settings have given some improvements • Use incremental GC rather than ‘stop the world’ style • Uses the flags that control survivor ratio for the GC generations – we have short lived objects • Increased default memory allocation .
but and average that is much higher • Problem appears to be in the multi-treading of Broker • Tried JNI against C++ broker and see same results .Did we make it? • First tests showed consistently slower times than the C+ + version ( up to 33* slower) • JProbe analysis and JVM tuning reduced this to about 2* slower on average • Through-put is slightly lower • JMeter shows a mean that is much faster than C++.
actual time start to finish 2. actual time spent on project 10 months) .Lessons learned • • • • • Always build a test suite before refactoring Never optimize until you know what to optimize Do not assume GC defaults are optimal Java needs lots of memory! Allow for distractions in your project estimates (estimated time 9 months.5 years.
Tech Lounge .
Project Goals • • • • • Nice. easy to use documentation store Replace mainframe based system Provide better editing and display options Allow limited access to subsets of articles Control the life-time of articles to ensure relevancy • Provide good search capabilities and crossreferencing options • Store multiple document formats .
dates. related articles.Design Choices • Store articles in native XML database • Article content stored against a subset of XHTML strict DTD – easy to display and construct • Each article has a meta-data XML file for owners. etc • Article display via XSLT for HTML and pdf • Article maintenance via jsps and custom tag libraries • tech lounge . number of reads.
Article Maintenance Architecture New Article jsp Maintain Article jsp Controller servlet Auth Task Form Parser Article Bean mainframe XML store .
Useful Tools • Many things from Apache • DHTML • Graphic designer Created Tools/utilities • Tamino Wrappers – abstraction layer • UID (QUID-like foreign key generator) .
Problems/Mistakes • Not using CVS well during first half of project • Not writing unit tests • Not using logging for debug messages .
Evolution • Display of XML – moving towards all XSLT • JSP usage – moving away from interlinked pages towards controller server style • JSP structure – scriplets bad. custom tags good(?) • Building up institutional tools/utilities and standards that are increasing productivity • Individual experts towards wide spread Java use through in-house training courses .
increased elegance • Many very powerful tools/utilities freely available Cons: • Requires higher competency of developers • Less productive for our typical applications • Needs larger machines .Review of our Java Experience Pros: • More flexible and capable • Greater abstraction possible.
Other projects • PdaParser – code generation to ease communication with mainframe modules • Fatcookie – authentication • Virtual File System extension to JEdit to work with our unix server script editing proceedure Current • Jar management database (entirely XML/XSLT) • Web service wrapper for mainframe modules • Dynamic pdf generation from mainframe data .
Useful Project Management Tip Remember the Bullfighter! .
This action might not be possible to undo. Are you sure you want to continue?
We've moved you to where you read on your other device.
Get the full title to continue reading from where you left off, or restart the preview.