GTLAB Installation Tutorial for SciDAC 2009

1. Installing and Building
GTLAB
Marlon Pierce
Indiana Univeristy

2. GTLAB and OGCE
OGCE contains multiple sub-projects
Portlet-based Grid portal (with GridSphere and Tomcat).
Workflow suite (services and add-ins to the portal)
Information Web services
Gadget container
Cyberaide JavaScript libraries
GTLAB
These are packaged with Maven and include everything you need
except Java and (for some services) MySQL.
We try to make things installable with minimal fuss.
Consistent directory structures across projects
Edit one config file (pom.xml)
Run one command (mvn clean install)
You may need to futz a little with MySQL

3. Overview and Examples

4. User’s Browser
Portlet, Gadget
Containers
iGoogle, Facebook
GTLAB Gadgets
TeraGrid Services
(GRAM, GridFTP, INCA)
Google GData Services,
Twitter Feeds, etc
RPS, RDS, GPIR
Services
GTLAB’s Relationship to Other
Components

5. Various GTLAB
applications deployed
as portlets:
Remote directory
browsing, proxy
management, and
LoadLeveler queues.

6. GTLAB Applications as Google Gadgets:
MOAB dashboard, remote directory
browser, and proxy management.

7. MyProxy Example
<h:form id="form">
<h:outputText value="Username:"/>
<h:inputText value="#{resource.username}" required="true"/>
<h:outputText value="Password: "/>
<h:inputSecret value="#{resource.password}" required="true"/>
<h:outputText value="MyProxy Server: "/>
<h:inputText value="#{resource.myproxyHostname}" required="true"/>
<o:submit id="submit" value="Submit" binding="#{builder.body}"
action="#{builder.junkAction}" >
<o:multitask id="multi" persistent="true" taskname="#{resource.taskname}">
<o:myproxy id="mypr" hostname="#{resource.myproxyHostname}"
lifetime="2" password="#{resource.password}" port="7512"
username="#{resource.username}"/>
</o:multitask>
</o:submit>
</h:form>

8. Executing Multiple Task
<o:submit id="submit" value="Submit" binding="#{builder.body}"
action="#{builder.junkAction}" >
<o:multitask id="multi" persistent="true"
taskname="#{resource.taskname}">
<o:jobsubmit id="js" myfaces="true"
arguments="#{resource.arguments}"
executable="#{resource.executable}"
hostname="#{resource.hostname}"
provider="#{resource.provider}"
stdout="#{resource.stdout}"/>
<o:filetransfer id="ft" myfaces="true"
from="#{resource.from}" to="#{resource.to}" />
<o:dependency id="dep" task="ft" dependsOn="js"/>
</o:multitask>
</o:submit>

9. <o:submit id="submit" value="Submit"
binding="#{builder.body}"
action="#{builder.junkAction}">
<o:multitask id="multi"
persistent="true"
taskname="#{resources.taskname”>
<o:rpsOptimal
resource="task1-resources"
id="rpsoptimal-exec"
myfaces="true"/>
</o:multitask>
</o:submit>
Resource Prediction Service
Client with GTLAB

10. Example with Dependency
<o:submit id="submit” …>
<o:multitask id="multi” …>
<o:rpsAddModel resource="task1-resources"
id="rpsAddModel-exec"
myfaces="true"/>
<o:rpsManager resource="task1-resources"
id="rpsManager-exec"
myfaces="true"/>
<o:dependency id="deps-rpsstuff”
task="rpsManager-exec"
dependsOn="rpsAddModel-exec"/>
</o:multitask>
</o:submit>

11. More Detail: Available
Tags and Features

12. GTLAB Features
Extends Java Server Faces.
oTag components wrap major COG Abstraction Layer features
oAnd Web Service clients, GridShib, etc.
oSupports tag inter-dependencies.
Allows you to do standalone development.
Use JSF portlet bridge to convert into portlets
No new coding, just add/modify XML config files and jars.
Process is automated
Or forget about portlets
oConvert into Google Gadgets
oDevelop as a Facebook application
oEtc.

13. Getting GTLAB
See http://www.collab-ogce.org/ogce/index.php/GTLAB
You can use your favorite SVN client to check out.
 svn co https://ogce.svn.sourceforge.net/svnroot/ogce/GTLAB
(latest)
svn co
https://ogce.svn.sourceforge.net/svnroot/ogce/tags/GTLAB-tg09
(tagged)
“Latest” will give you easy access to any updates
“svn update”
Best option if you want to actively develop and get fixes right
away.

14. No SVN? Get the TAR
SourceForge’s SVN/CVS
viewer now provides a
“Download GNU Tar”
option.
http://ogce.svn.sourceforge.net/viewvc/ogce/GTLAB/
for latest.
http://ogce.svn.sourceforge.net/viewvc/ogce/tags/GTLAB-tg09/
for tag

15. Levels of Abstraction in GTLAB code
You can use prepackaged gadgets/portlets.
You can develop new applications using
existing tag libraries.
You can make new tags for your services
You can hack or steal the code
Note JSF is compatible with JSP, so you can mix and
match.
And you can embed JavaScript
YUI, Scriptaculous user interface goodies.
Google and Facebook APIs.

16. Tag Description
MyProxy Gets a proxy credential (COG)
JobSubmit Runs remote commands (COG)
FileOperation Create, delete, list remote files and
dirs (COG)
FileTransfer GridFTP downloads, 3
rd
party
transfers (COG)
XSubmissionOutputFormat outputs of job and file
operations
ResourceDiscoveryClients for the RDS
ResourcePredictionClients for the RPS
MyCommunityProxyGet and decorate TG Community
credentials with Grid Shib
Community Log Log use of TG Community credentials

17. GTLAB Example Description
MyProxyExample Get a credential and store in OGCE
global proxy store.
JobSubmitExamplesVarious examples showing how to
submit a grid job and get output.
Dashboard ExamplesRuns showq on Big Red, Cobalt, and
Mercury and formats output.
DirectoryBrowser Displays and downloads files on a
remote host via GridFTP
MultipleTaskExampleShows how to couple file transfers,
operations, and
These will work as gadgets, portlets,
or standalone applications

18. Gadgets and Portlets

19. GTLAB Gadgets
Code
Base
GTLAB Portlets
Code
Base
Tomcat
Web
Server
Tomcat Web Server
+
Portlet Container
iGoogle
Client
Aggregator
Compile and deploy into server
Decorate with bridge
and container jars, xml
config files.
GTLAB gadgets
can also run as
portlets with no
code changes by
using the JSF
portlet bridge.

20. Making GTLAB Google Gadgets
A Gadget is a standalone application that is integrated
into your personal iGoogle display.
They can run on your Web server.
You can use SSL/HTTPS
Google Gadgets have two versions
http: these use Google’s JavaScript APIs
url: these wrap external applications with Iframes.
We currently support “url” gadgets.

21. Example Gadget Config File
<?xml version="1.0" encoding="UTF-8" ?>
<Module>
<ModulePrefs title="MyProxy Gadget Example"
scrolling="true"
height="500"/>
<Content type="url"
href="http://…/:8080/GTLAB/examples/MyProxyExample.jsf">
</Content>
</Module>
Save this as MyProxy.xml and place
in a Web accessible place.

22. Other Gadgets
Providers
Tomcat + GTLAB
Gadgets
Grid and Web
Services
(TeraGrid, OSG, etc)
Other Gadgets
Providers
Social Network
Services (Orkut,
LinkedIn,etc)
RSS Feed, Cloud, etc
Services
Gadget containers
aggregate content
from multiple
providers. Content
is aggregated on the
client by the user.
Nearly any web
application can be a
simple gadget (as
Iframes)

23. Click “Add stuff” and
then “Add feed or
gadget”. Type the
URL of your gadget.

24. GTLAB Applications as Google Gadgets:
MOAB dashboard, remote directory
browser, and proxy management.

25. Tomcat
+
Portlets and
Container
Grid and Web
Services
(TeraGrid, OSG, etc)
Grid and Web
Services
(TeraGrid, OSG, etc)
Grid and Web
Services
(TeraGrid, OSG, etc)
HTML/HTTP
SOAP/HTTP
Common science
gateway
architecture.
Aggregation is in
the portlet
container.
Users have limited
selections of
components.

26. Making GTLAB Gadgets into Portlets
Do all development in jsf_standalone directory
This is where your template tags will be placed
When you are ready to convert:
Download and build the OGCE portal
cd GTLAB/transition
mvn clean process-resources
cd GTLAB/portlets
mvn clean install
You will need to make small edits to portlet.xml,
group.xml, and layout.xml in
GTLAB/portlet/src/main/webapp/WEB-INF/
That’s it.

27. Various GTLAB
applications deployed
as portlets:
Remote directory
browsing, proxy
management, and
LoadLeveler queues.

28. Getting Started with
GTLAB

29. GTLAB Directory Structure

30. Build GTLAB
Unpack or checkout code
Cd GTLAB
All commands are
executed from here.
Edit properties at the
top of pom.xml.
Change IP
Change project.home if
you unpack someplace
besides $HOME.
Run “mvn clean install”
<properties>
<portal.server.ip>
156.56.104.143
</portal.server.ip>
<host.base.url>
http://${portal.server.ip}:8080/
</host.base.url>
<project.home>
${env.HOME}/GTLAB
</project.home>
<tomcat.version>
apache-tomcat-5.5.27
</tomcat.version>
<catalina.home>
${project.home}/portal_deploy/${tomcat.version}/
</catalina.home>
<dot.globus.home>
${env.HOME}/.globus/
</dot.globus.home>
</properties>

31. Run Examples
From GTLAB, start tomcat with ./startup.sh.
From GTLAB, stop Tomcat with ./shutdown.sh
 Point browser to http://localhost:8080/GTLAB
Start with MyProxy Example

32. Next Steps
Play with examples.
These are really bare bones. Make something
interesting.
Make a Google gadget.
Mix and match tags in a pipeline to make a new
application.
Use the dependency tag.
Note you can mix and match JSF and JSP if you are
not familiar with JSF.
Try making a new tag.
Explained next.

33. Making New Tags and
Examples

34. Making a New JSF Page from Tags
I recommend starting from the examples.
jsf_standalone/src/main/webapp/examples
“Build” the examples with
mvn –o clean install –f jsf_standalone/pom.xml
The “-o” option is used to build offline. Will also avoid
unnecessary Maven repository updates.
The “-f” specifies only build this specific module.
I recommend not futzing with the deployed versions under
portal_deploy.
A computer is a state machine. State must be reproducible.

35. Making a New Tag
Run the following command from GTLAB:
mvn clean process-resources -Dtag.name=test
-Dprojectname=Test -f templateTag/pom.xml
Add -Ddest.dir=/tmp for a dry run.
Replace “test” with the name of your tag.
Replace “Test” with the name for your Bean.
This will make 4 files
TestBean.java, TestTag.java, TestBeanFactory.java, UITest.java
Edits also 3 config files
gtlab-factory.xml, managed-beans.xml, components.xml
This will compile but to implement something useful, you will
need to edit the highlighted files.

36. Implementing a Tag
The place to start is TestBean.java (or whatever
you used for –Dprojectname=…).
This includes several inherited methods that can be
implemented.
Most important is submit(). Use the try/catch
block. This is where the action is.
If you want to hook tags into chains, implement
getOutput() and setInput().
Also take a look at the other beans.

37. Suggested Tags: A Wish List
What can you do in your bean? Anything server-side Java
can do.
Some suggestions:
Implement a tag client to a remote Web service. Amazon has
some interesting ones….
Implement an RSS/Atom feed client to Twitter, your blog,
Facebook, etc.
Combine the feeds as a mash-up.
Connect to a database with JDBC.
Implement a JMS publisher or subscriber.
Use Google Java APIs to interact with Blogger, Calendar, and
YouTube.
Try interacting with Facebook.

38. INCA Tags: An Extended
Example

39. Example: Make an INCA Dashboard
INCA is used as the testing framework for the TeraGrid.
You can get the latest INCA test results from the URL
http://inca.teragrid.org/inca/XML/kit-status-v1/portal_summary
See Sangmi’s notes at
http://sangpall.blogspot.com/2009/03/being-nice-client-of-inca-service.html
You’ll get back some fine looking XML.
Let’s make a little dashboard out of this.

40. First, Create a New, Empty Tag
mvn clean process-resources \
-Dtag.name=incastatus \
-Dprojectname=IncaStatus \
-f templateTag/pom.xml

41. Second, Implement the Bean
Main thing is to implement IncaStatusBean’s
submit() method.
Code is hard to put in a PPT slide
See
http://ogce.svn.sourceforge.net/viewvc/ogce/GTLAB/jsf_standalone/src/main/java/ogce/gsf/gridbeans/IncaStatusBean.java

42. Third, Implement a JSF Page
Actual tag is shown below.
Full example is
http://ogce.svn.sourceforge.net/viewvc/ogce/GTL
AB/jsf_standalone/src/main/webapp/examples/Inc
aStatusExample.jsp?view=log
It includes some boilerplate (can be automated).
<o:incastatus id=”mytest"
resource="task1-resources”
myfaces="true"/>

43. Here’s the resulting page. You can make into a portlet in a
couple of steps. You can also tie to other tags in a
pipeline.

44. Next Steps: Make a Pipeline
Implement the
setInput() and
getOutput() methods.
Pipeline it with the
Resource Description and
Resource Prediction
Service tags.
Integrate QBETS tags
Currently unwritten
Integrate with Job
Submission
Find Running Hosts
(INCA or RDS)
Find Best Host for
YourJob (RPS)
Find Host with
Shortest Wait Time
(QBETS)
Submit Job

46. More Stuff about GTLAB
Supports linear tag dependencies.
Removed dependence on COG graph processing (still use the
COG)
We can support graphs of any tag you care to write.
Only support pipelines currently, not full DAGs
Redesigned to simplify writing new tags.
Extensive use of factories.
You no longer have to change core code to add a tag
Templates for 4 Java files and 3 XML config files now can be
done in one (mvn) step
Typically you will only need to edit one Java file.