You are on page 1of 69

Integrating PI

System with R
PI DEVELOPERS CLUB WHITE PAPER

OSISOFT, LLC

How to Contact Us
Worldwide Offices
Email: pidevclub@osisoft.com
Web: pisquare.osisoft.com/community/developers-club

OSIsoft Australia
Perth, Australia
Auckland, New Zealand

OSIsoft Europe
OSIsoft, Inc.

Altenstadt, Germany

777 Davis St., Suite 250


San Leandro, CA 94577 USA

OSI Software Asia Pte Ltd.

Houston, TX
Johnson City, TN
Mayfield Heights, OH
Phoenix, AZ
Savannah, GA
Seattle, WA
Yardley, PA

Singapore

OSIsoft Canada ULC


Montreal, Quebec
Calgary, Alberta

OSIsoft, Inc. Representative Office


Shanghai, Peoples Republic of China

OSIsoft Japan KK
Tokyo, Japan

OSIsoft Mexico S. De R.L. De C.V.


Mexico City, Mexico

WWW.OSISOFT.COM
OSIsoft, Inc. is the owner of the following trademarks and registered trademarks: PI System, PI
ProcessBook, Sequencia, Sigmafine, gRecipe, sRecipe, and RLINK. All terms mentioned in this book
that are known to be trademarks or service marks have been appropriately capitalized. Any trademark
that appears in this book that is not owned by OSIsoft, Inc. is the property of its owner and use herein
in no way indicates an endorsement, recommendation, or warranty of such partys products or any
affiliation with such party of any kind.
RESTRICTED RIGHTS LEGEND
Use, duplication, or disclosure by the Government is subject to restrictions as set forth in
subparagraph (c)(1)(ii) of the Rights in Technical Data and Computer Software clause at DFARS
252.227-7013
Unpublished rights reserved under the copyright laws of the United States.

1998-2016 OSIsoft, LLC

TABLE OF CONTENTS

Overview ..................................................................................................................................... 5
About this Document ....................................................................................................................... 5
About this White Paper .................................................................................................................... 5
What You Need to Start ................................................................................................................... 5
1. Introduction .......................................................................................................................... 6
What Is R?......................................................................................................................................... 6
What can R do best?......................................................................................................................... 6
Why integrating the PI System with R? ............................................................................................ 6
How to use this white paper?........................................................................................................... 7
2. Integrating R and .NET ........................................................................................................... 8
What is R.NET? ................................................................................................................................ 8
Installing R.NET ................................................................................................................................. 8
3. Programming with R.NET ..................................................................................................... 11
How to initialize R.NET objects? ..................................................................................................... 11
Evaluate method ............................................................................................................................ 11
R.NET Data Types ........................................................................................................................... 11
Importing objects from R ............................................................................................................... 13
Exporting objects to R .................................................................................................................... 14
4. Integrating R and Java .......................................................................................................... 16
What is JRI? .................................................................................................................................... 16
5. Programming with JRI .......................................................................................................... 19
How to initialize JRI objects? .......................................................................................................... 19
JRI Data Types ................................................................................................................................. 22
Importing objects from R ............................................................................................................... 22
Exporting objects to R .................................................................................................................... 24
Using JavaGD on your Java projects ............................................................................................... 25
3

6. R Console and Functions ...................................................................................................... 26


Installing R Packages....................................................................................................................... 26
R Functions ..................................................................................................................................... 26
7. Getting Data from the PI System .......................................................................................... 35
PI AF SDK......................................................................................................................................... 35
PI JDBC ............................................................................................................................................ 36
PI Web API ...................................................................................................................................... 37
8. Integrating R and other systems ........................................................................................... 39
9. Developing Applications....................................................................................................... 40
Application with GUI to generate graphics .................................................................................... 40
Developing a PI ACE Library with R.NET ......................................................................................... 43
Developing custom interfaces ........................................................................................................ 45
10. Retrieving PI Data directly into R .......................................................................................... 47
Creating .NET libraries on top of PI AF SDK for using wih rClr ....................................................... 47
Making RESTful WebService calls against PI Web API directly from R using rCurl ........................ 50
Retrieving data from the PI System using RODBC and PI ODBC. ................................................... 52
11. Developing the R Multi-Correlation PI Coresight custom symbol ........................................... 55
Developing the custom ASP.NET Web API project with R.NET ...................................................... 55
Developing the custom PI Coresight symbol .................................................................................. 63
12. Conclusions ......................................................................................................................... 67
References................................................................................................................................. 68
Revision History ......................................................................................................................... 69

OVERVIEW
ABOUT THIS DOCUMENT
This document is available as part of the PI Developer Club website content, under the PI
Developers Club White Paper and Tutorials category, here.
Any question or comment related to this document should be posted in the appropriate PI
Developers Club discussion forum or sent to the PI Developers Club Team at
pidevclub@osisoft.com.

ABOUT THIS WHITE PAPER


The purpose of this white paper is to provide information to the PI user about how to
integrate the PI System with R through R.NET for .NET development and JRI for Java
development.

WHAT YOU NEED TO START


The following pieces were used to implement the examples and methods proposed in this white
paper:

PI Data Archive 2016


PI System Management Tools (SMT) 2016
PI SDK 1.4.4
PI AF SDK and PI System Explorer 2016
PI Web API 2016
PI JDBC 2016
PI ACE 2010 R2
R Statistical Computing (R console) 3.3.1
Microsoft Visual Studio 2013/2015
R.NET 1.6.5 (package for .NET development)
Java 8 Update 91
Eclipse Neon IDE for Java Developers
JRI (package for Java development)

It is always recommended to use the most recent version of the products above.

1. INTRODUCTION
WHAT IS R?
R is a free software programming language and software environment for statistical computing and
graphics. The R language is widely used among statisticians and data miners for developing statistical
software and data analysis. Polls and surveys of data miners are showing R's popularity has increased
substantially in recent years.
R is an implementation of the S programming language combined with lexical scoping semantics
inspired by Scheme. S was created by John Chambers while at Bell Labs. R was created by Ross Ihaka
and Robert Gentleman at the University Of Auckland, New Zealand, and is currently developed by the
R Development Core Team, of which Chambers is a member. R is named partly after the first names
of the first two R authors and partly as a play on the name of S.
R is a GNU project. The source code for the R software environment is written primarily in C, Fortran,
and R. R is freely available under the GNU General Public License, and pre-compiled binary versions
are provided for various operating systems. R uses a command line interface; however, several
graphical user interfaces are available for use with R.
R performs complex data science and it is used by over two million analysts. R has become the tool of
choice for the worlds best statisticians, data scientists and analysts.

WHAT CAN R DO BEST?


R provides a wide variety of statistical techniques, including linear and nonlinear modeling, classical
statistical tests, time-series analysis, classification, clustering, and others. R can also handle regression
and correlation analysis not to mention machine learning. Finally, it provides graphical features to plot
great visuals.

WHY INTEGRATING THE PI SYSTEM WITH R?


There is a successful story about integrating R and Hadoop infrastructure, which is a software
framework for storage and large-scale processing of data-sets, in a Big Data Workshop held by
the Boston Predictive Analytics group (please refer to this article). This shows that R can make sense
with big data.
The PI System is a suite of software products that are used for data collection, historicizing, finding,
analyzing, delivering, and visualizing. It is marketed as an enterprise infrastructure for management
of real-time data and events. It provides a lot of efficient different PI System data access products in
order to access data stored on the PI Data Archive.
Integrating the PI System with R lets you perform advanced statistical analytics and calculations with
your PI System data, having the possibility to show the results on a nice graphic or storing them back
to the PI System.
As C# and Java are two very popular coding languages, this white paper has focused on JRI, which is a
Java library to communicate with R, and R.NET, which does the same but for .NET Framework.

HOW TO USE THIS WHITE PAPER?


The following paragraphs provides information about how this white paper is structured.
Chapter 1 provides information about what is R, what it can do best and describe the benefits of
integrating the PI System with this product.
Chapter 2 and 3 are about the setting up and using the R.NET library in your custom application
developed in Visual Studio in order to integrate the PI System with .NET Framework. You can skip
those chapters if you are developing against Java.
Chapter 4 and 5 are about the setting up and using the JRI library in your custom application developed
in Java/Eclipse in order to integrate the PI System with Java. You can skip those chapters if you are
developing against .NET Framework.
Chapter 6 is about creating R functions which are going to be used by your custom application.
Chapter 7 provides information about the recommended PI System Access products in order to
retrieve data from the PI System.
Chapter 8 shows other alternatives of developing a custom application if you are not using .NET
Framework neither Java.
Chapter 9 shows some interesting use cases that could be referred to when developing your own
custom application.
Chapter 10 shows alternatives to retrieve PI data into R directly, which many data scientists would
prefer.
Chapter 11 shows how to create a custom PI Coresight symbol for displaying multi-correlations for
multiple attributes.
Chapter 12 is about the conclusions of this white paper.

2. INTEGRATING R AND .NET


WHAT

IS

R.NET?

R.NET enables .NET Framework to collaborate with R statistical computing (R console). R.NET requires
.NET Framework 4 and native DLLs installed with R environment. You need no other extra
installations. As a result, you need to have a machine with Visual Studio 2010 or higher installed in
order to develop a project with this library. This version of R.NET does not require .NET Framework
4.5, but this product needs to be installed in case Visual Studio 2012 is your Integrated Development
Environment.

INSTALLING R.NET
How to download R.NET?

The version R.NET 1.6.5 or higher could be downloaded through NuGet Gallery.
Installing NuGet
NuGet is a free and open source package manager for the .NET Framework, which could be installed
and updated using the Visual Studio Extension Manager. To check if your copy of Visual Studio already
has the NuGet extension, look for Library Package Manager in the Tools menu of your copy of Visual
Studio.

Figure 1 Executing Package Manager Console.


8

If your copy of Visual Studio does not already have the Library Package Manager (NuGet) extension,
you can install it using the Extension and Updates.
Using the Extension and Updates

In Visual Studio, click Tools and then Extension and Updates.


Then, select the Online Visual Studio Gallery tab, and enter "nuget package manager" in the
search box to find the NuGet Package Manager extension.
Select NuGet Package Manager and then click Download.

Figure 2 Installing NuGet Package Manager

In the Installer dialog box, click Install.

Figure 3 License Terms for NuGet Package Manager.


When installation is complete, you should close and re-open Visual Studio.

Figure 4 Installation complete dialog for NuGet Package Manager.

After NuGet is installed properly, you can transfer the R.NET library from the internet running the
following command in the Package Manager Console: Install-Package R.NET.Community.
The R.NET libraries will be added automatically to your Visual Studio project.

10

3. PROGRAMMING WITH R.NET


HOW TO INITIALIZE R.NET OBJECTS?
After you have added the R.NET library to your Visual Studio project, make sure to add the
C:\Program Files\R\R-3.3.1\bin\i386 folder to the PATH system environment variable. You can
execute the command @echo %PATH% to find out its current status.
The way to initialize the R.NET is very simple:
C#
engine = REngine.GetInstance();
engine.Initialize();

EVALUATE METHOD
The evaluate method is the one of the most important methods on R.NET, since the string sent, which
is the unique parameter of this method, is actually the R code that will be executed on R console. For
instance, if you want to create a new array variable (through R.NET) on memory that stores all the
values from 1 to 100, you just need one line of code:

R
x<-1:100

C#
engine.Evaluate("x<-1:100)");

R.NET DATA TYPES


All expressions in R are represented in SymbolicExpression class in R.NET. For data accession, the
following special classes are defined. Please refer to Table 1 for additional information.

R.NET

.NET Framework

character
vector

RDotNet.CharacterVector System.String[]

Note

11

integer
vector

RDotNet.IntegerVector

System.Int32[]

real vector

RDotNet.NumericVector

System.Double[]

complex
vector

RDotNet.ComplexVector

System.Numerics.Complex[]

raw vector

RDotNet.RawVector

System.Byte[]

logical
vector

RDotNet.LogicalVector

System.Boolean[]

character
matrix

RDotNet.CharacterMatrix System.String[, ]

integer
matrix

RDotNet.IntegerMatrix

System.Int32[, ]

real
matrix

RDotNet.NumericMatrix

System.Double[, ]

complex
matrix

RDotNet.ComplexMatrix

System.Numerics.Complex[,
]

raw
matrix

RDotNet.RawMatrix

System.Byte[, ]

logical
matrix

RDotNet.LogicalMatrix

System.Boolean[, ]

list

RDotNet.GenericVector

From version 1.1.

data
frame

RDotNet.GenericVector

From version
1.1. RDotNet.DataFrame class is
also available (below).

data
frame

RDotNet.DataFrame

From version 1.3.

function

RDotNet.Function

From version 1.4. Including


closure, built-in function, and
special function.

factor

RDotNet.Factor

S4

RDotNet.S4Object

System.Int32[]

The minimum value in R is 2^31+1 while that of .NET


Framework is -2^31.

System.Numerics assembly is
required for .NET Framework 4.

The minimum value in R is 2^31+1 while that of .NET


Framework is -2^31.

System.Numerics assembly is
required for .NET Framework 4.

From version 1.5.2.


Not Available Yet. See S4 branch
in the source control.

12

Table 1 - Classes in R.NET to link R and .NET Framework.

IMPORTING OBJECTS FROM R

Example 1 Converting R integer variable to R.NET IntegerVector and integer array

C#
RCodeString = "myintvector<-1:10";
Console.WriteLine("\nR Code: " + RCodeString);
IntegerVector myIntegerVector = engine.Evaluate(RCodeString).AsInteger();
int[] myIntegerArray = myIntegerVector.ToArray();
Console.WriteLine("\nInteger Vector: ");
i = 1;
foreach (int myInteger in myIntegerVector)
{
Console.WriteLine(i + " value=" + myInteger);
i++;
}

Example 2 Converting R real variable to R.NET NumericVector and double array

C#
RCodeString = "myrealvector<-rnorm(5, 0, 1)";
Console.WriteLine("\nR Code: " + RCodeString);
NumericVector myNumericVector = engine.Evaluate(RCodeString).AsNumeric();
double[] myDoubleArray = myNumericVector.ToArray();
Console.WriteLine("\nNumeric Vector: ");
i = 1;
foreach (double myNumeric in myNumericVector)
{
Console.WriteLine(i + " value=" + myNumeric);

i++;

Example 3 Converting R complex variable to R.NET ComplexVector and complex array

C#
RCodeString = "mycomplexvector<- 1:2 + 1i*(8:9)";
Console.WriteLine("\nR Code: " + RCodeString);
ComplexVector myComplexVector = engine.Evaluate(RCodeString).AsComplex();
Complex[] myComplexArray = myComplexVector.ToArray();

13

Console.WriteLine("\nComplex Vector: ");


i = 1;
foreach (Complex myComplex in myComplexVector)
{
Console.WriteLine(i + " value=" + myComplex.Imaginary + "i + " +
myComplex.Real);
i++;
}

Example 4 Converting R logical variable to R.NET LogicalVector and boolean array

C#
RCodeString = "mylogicalvector<-c(FALSE, TRUE, FALSE, TRUE, FALSE)";
Console.WriteLine("\nR Code: " + RCodeString);
LogicalVector myLogicalVector = engine.Evaluate("c(FALSE, TRUE, FALSE,
TRUE, FALSE)").AsLogical();
Boolean[] myBooleanArray = myLogicalVector.ToArray();
Console.WriteLine("\nLogical Vector: \n");
i = 1;
foreach (Boolean myBoolean in myLogicalVector)
{
Console.WriteLine(i + " value=" + myBoolean.ToString());
i++;
}

EXPORTING OBJECTS TO R
Example 1 Converting string array to R.NET CharacterVector and R variable

C#
Console.WriteLine("\nR character vector\n");
string[] myStringArray = new string[] {
"PIDataLink","PIProcessBook","PIWebParts" };
CharacterVector myCharacterVector =
engine.CreateCharacterVector(myStringArray.AsEnumerable());
engine.SetSymbol("mycharvector", myCharacterVector);
engine.Evaluate("print(mycharvector)");

Example 2 Converting double array to R.NET NumericVector and R variable


14

C#
Console.WriteLine("\nR real vector\n");
NumericVector myNumericVector = engine.CreateNumericVector(new double[] {
30.02, 29.99, 30.11, 29.97, 30.01, 29.99 });
engine.SetSymbol("mynumvector", myNumericVector);
engine.Evaluate("print(mynumvector)");

Example 3 Converting byte array to R.NET RawVector and R variable

C#
Console.WriteLine("\nR raw vector\n");
byte[] myByteArray = System.Text.Encoding.ASCII.GetBytes("u03a0");
RawVector myRawVector = engine.CreateRawVector(myByteArray);
engine.SetSymbol("myrawvector", myRawVector);
engine.Evaluate("print(myrawvector)");

15

4. INTEGRATING R AND JAVA


In this chapter, it is shown how to develop a Java application which communicates with R through JRI.
We are going to show not only how to install JRI but also all the software required to run this
application (Java and Eclipse). One interesting advantage of using Java is that it is compatible not only
with Windows but also with many different operating systems (OS) as Linux. Although all the code
snippets of this white paper were tested on Windows, the same code could be used on different
platforms.

WHAT IS JRI?
JRI is a Java/R Interface which allows you to run R inside Java applications as a single thread. It
loads the R dynamic library into Java and provides a Java API to R functionality. In a sense JRI is the
inverse of rJava and both could be combined (i.e. you can run R code inside JRI that calls back to the
JVM via rJava).
Installing JDK on Windows
Detailed instructions on how to install the JDK (Java Development Kit) for your particular operating

system are available from the JDK download website. As I have only installed JDK on Windows, the
instructions below are for the Microsofts operating system.
Remember that the JDK and the documentation are separate, and you install them separately. If you
are pushed for disk space, you don't have to install the documentation because you can access it
online. The link for the current location of the online documentation for the JDK could be found here.
After downloading and installing the latest version of the Java JDK from Oracle, you need to make sure
that if you open the command prompt, the file javac.exe is found. This can be achieved by adding the
bin folder of the JDK directory (C:\Program Files\Java\jdk1.7.0_40\bin in my case) to the PATH system
environmental variable as described in this page.
Installing Eclipse
Most people know Eclipse as an integrated development environment (IDE) for Java. Today it is the
leading development environment for Java with a market share of approximately 65%.

Eclipse is created by an Open Source community and is used in several different areas because it is a
development environment for PHP, Python, Java, Android applications and other languages.
The Eclipse IDE can be extended with additional software components. Eclipse calls these software
components plug-ins. Several Open Source projects and companies have extended the Eclipse IDE or
created standalone applications (Eclipse RCP) on top of the Eclipse framework.
In order to install Eclipse, please visit the Eclipse Downloads Page.

16

Installing JRI

The recommended way to install JRI is through R. JRI comes bundled with rJava, so the best way is to
simply install rJava on the R console. It is available from CRAN, so open the R console and execute:
R
install.packages("rJava")

After running the command above rJava will be installed. A new folder called rJava is created on the
%R_HOME%\library folder. There is also a new folder called JRI under the rJava folder with all the jar
files that you need to import on your Java project.
At this point, you should have the following system environmental variable properly configured on
your system for 32-bit Windows OS. If you are using the 64-bit version of Windows, you should replace
the i386 folder to the x64 folder.

System Environmental Variable

Value

R_HOME

C:\Program Files\R\R-3.3.1
C:\Program Files\Java\jdk1.7.0_40\bin;
C:\Program Files\R\R-3.3.1\library\rJava\jri\i386;

PATH

C:\Program Files\R\R-3.3.1\bin\i386;
C:\Program Files\R\R-3.3.1\library\rJava\jri;

Table 2 - Required System Environmental Variables for Java on a 32-bit Windows OS.

JavaGD

JavaGD is a package for R (see www.r-project.org) providing a Java Graphics Device, which is a device
delegating all painting functions of R to a Java class. The default implementation provides an object
of the class Canvas, which can be used in any Java application. Currently the main use of JavaGD is in
the JGR project, but it doesn't depend on it.
Installing JavaGD should just be a matter of typing install.packages("JavaGD") at your R console and
waiting - have a look at the JavaGD homepage. If errors occur, the same holds as in "Installing JRI".
You have to carefully check that all that is needed is on the LD_LIBRARY_PATH, and that you have
Java installed and all necessary paths set up correctly.

17

There is an example below. Enter these commands at your R console and you will plot a figure to a
JavaGD device. If this doesn't work, something has gone wrong with your installation. In my case, it
has only worked on R 3.3.1 x64 version (the i386 version does not work).
R
library(JavaGD)
JavaGD()
plot(c(1,5,3,8,5), type='l', col=2)

18

5. PROGRAMMING WITH JRI


HOW TO INITIALIZE JRI OBJECTS?
Please follow the procedure below in order to create your first project that initialize JRI objects to
communicate with R binaries:
1. Open Eclipse and create a new project on Eclipse called FirstJRIProgram.
2. Create a new package called TestingJDRI.
3. Right-click the new project and select Properties. Click on Java Build Path on the left and then
select the Libraries ribbon.
4. Click on the button Add External JARs and select the file JRI.jar located under the folder
%R_HOME%\library\rJava\jri as shown on Figure 5.

Figure 5 Adding external JARs to your Java Project on Eclipse.

5. Create a new class called Example1 under the TestingJRI package.


6. Copy and paste the code snippet below:

package TestingJRI;
import org.rosuda.JRI.Rengine;

public class Example1 {


public static void main(String[] args) {
System.out.println("Creating Rengine (with arguments)");
// If not started with --vanilla, funny things may happen in this R
// shell.

19

String[] Rargs = { "--vanilla" };


Rengine re = new Rengine(Rargs, false, null);
System.out.println("Rengine created, waiting for R");
// the engine creates R is a new thread, so we should wait until it's
// ready
if (!re.waitForR()) {
System.out.println("Cannot load R");
return;
}
System.out.println("Finished loading R");
re.end();
}
}

In case you are running a 64 bit Java Application, before you run your application, right click on the
project and select Run As Run Configurations.
Under
the
arguments
tab,
type
-Djava.library.path="C:\Program
3.3.1\library\rJava\jri\x64" on the VM arguments.

Files\R\R-

Figure 6 Graphic generated by R function PI_Histogram.

20

On the environment tab, create a variable called PATH with a value of C:\Program Files\R\R3.3.1\bin\x64.

Figure 7 Graphic generated by R function PI_Histogram.

Then, click on the Run button. You are supposed to see the following output on the Eclipse console:
Creating Rengine (with arguments)
Rengine created, waiting for R
Finished loading R

21

If your program is not working properly check if the system enviromental variables are well configured.

JRI DATA TYPES


The JRI data types are simpler than the R.NET data types since the JRI variable is REXP for most of
the used variables as string, integers, real and boolean arrays. Please refer to the table 3.

JRI

Java

Character vector

REXP

String[]

Integer vector

REXP

int[]

Real vector

REXP

double[]

Boolean vector

REXP

boolean[]

Table 3 - Classes in JRI to link R and Java.

IMPORTING OBJECTS FROM R


The code snippet below is similar to the previous chapter but using JRI and not R.NET. This is an
example about how to transfer data stored on R variables and store it on Java variables.
import org.rosuda.JRI.RBool;
import org.rosuda.JRI.REXP;
import org.rosuda.JRI.Rengine;

public class ImportingObjs {


public static void Start(Rengine re)
{
System.out.print("ImportingObjs");
String RCodeString ="";
//R string vector
RCodeString = "mycharvector<-c(\"hello\",\"bye\",\"OSIsoft\")";
System.out.println("\nR Code: " + RCodeString);
REXP Rstringvector = re.eval(RCodeString);
String[] myStringArray = Rstringvector.asStringArray();
System.out.println("\nmyStringArray: ");
int i = 1;
for(String myString : myStringArray)
{
System.out.println(i + " value: " + myString);

22

i++;
}

//R int vector


RCodeString = "myintvector<-1:10";
System.out.println("\nR Code: " + RCodeString);
REXP Rintvector = re.eval(RCodeString);
int[] myIntegerArray = Rintvector.asIntArray();
System.out.println("\nInteger Vector: ");
i = 1;
for(int myInteger : myIntegerArray)
{
System.out.println(i + " value=" + myInteger);
i++;
}

//R real vector


RCodeString = "myrealvector<-rnorm(5, 0, 1)";
System.out.println("\nR Code: " + RCodeString);
REXP Rrealvector= re.eval(RCodeString);
double[] myDoubleArray = Rrealvector.asDoubleArray();
System.out.println("\nNumeric Vector: ");
i = 1;
for(double myNumeric : myDoubleArray)
{
System.out.println(i + " value=" + myNumeric);
i++;
}

//R Boolean
RCodeString = "mybool<-c(FALSE)";
System.out.println("\nR Code: " + RCodeString);
RBool myBool = re.eval(RCodeString ).asBool();
System.out.println(i + " value=" + myBool.toString());

}
}

23

EXPORTING OBJECTS TO R
In order to transfer data from Java variables and stored them on R, please refer to the following
example.
import org.rosuda.JRI.Rengine;
import org.rosuda.JRI.REXP;

public class ExportingObjs {


public static void Start(Rengine re)
{

System.out.println("\n\nExporting Objects\n\n");
REXP x = null;
x=new REXP();
System.out.println(x.toString());
//R character vector -- R.NET RDotNet.RDotNet.CharacterVector
System.out.println("\nR character vector\n");
String[] myStringArray = new String[] {
"PIDataLink","PIProcessBook","PIWebParts" };
re.assign("mystringarray", myStringArray);
System.out.println(x=re.eval("mystringarray"));

//R integer vector


System.out.println("\nR int vector\n");
int[] myIntegerArray = new int[] { 4, 6, 10, 140, 54, 25 };
re.assign("myintarray", myIntegerArray);
System.out.println(x=re.eval("myintarray"));

//R real vector


System.out.println("\nR real vector\n");
double[] myDoubleArray = new double[] { 30.02, 29.99, 30.11, 29.97, 30.01,
29.99 };
re.assign("mydoublearray", myDoubleArray);
re.eval("print(mydoublearray)");
System.out.println(x=re.eval("mydoublearray"));

//R logical vector


System.out.println("\nR boolean vector\n");
boolean[] myBoolArray = new boolean[]{ true, false, false };
re.assign("myboolvector", myBoolArray);
System.out.println(x=re.eval("myboolvector"));

24

}
}

USING JAVAGD ON YOUR JAVA PROJECTS


It is not recommended to let R generate and control the graphics when using JRI because the graphics
might freeze. Nevertheless, this behavior does not happen for R.NET. That is why for Java applications,
it is better to call an implementation of JavaGD before plotting the graphics. The code snippet below
shows how to initialize the default implementation of JavaGD:

engine.eval("library(JavaGD)");
engine.eval("JavaGD()");

It is possible to draw your own implementation of JavaGD device (and not the default one). In this
scenario, the function JavaGD() calls the implementation of JavaGD found in the class defined by the
environment variable JAVAGD_CLASS_NAME. Hence, it is necessary to create a class that extends
GDInterface, set JAVAGD_CLASS_NAME to the name of your class. The easiest way of doing this is
having the Rengine evaluate "Sys.putenv('JAVAGD_CLASS_NAME'= 'MyJavaGD1')" as shown on the
code snippet below.

engine.eval("library(JavaGD)");
engine.eval("Sys.putenv('JAVAGD_CLASS_NAME'='MyJavaGD1')");
engine.eval("JavaGD()");

Finally, it needed to compile making sure that this time, you also have javaGD.jar on your classpath.
This file should be located on the library folder of R, since it was installed through the R console.

25

6. R CONSOLE AND FUNCTIONS


In this chapter, there are some examples of pure R functions that could be called from your Java or
.NET application. Those R functions could be stored on a specific file and loaded as soon as the R is
initialized so that their functions are ready to be called.

INSTALLING R PACKAGES
In order to use all the R functions described on this white paper, there are some packages that
should be downloaded from the internet. To download it, just open R console and type:
R
install.packages("packagename",lib=.Library)

R FUNCTIONS
Seven R functions were chosen to be used on this white paper. Data from PI Points shown on this
document are real values related to Power and Temperature.
PI Histogram
This function computes a histogram of the given data values. On the example below, it is shown the
histogram of a tag which is the outside temperature of OSIsoft San Leandro Office. This function lets
you choose the number breakpoints as well as the color of the graphic.

26

Figure 8 Graphic generated by R function PI_Histogram.

R
#Simple histogram - v is a vector of numercial values - breaks is the number of breaks
in the histogram
PI_Histogram <- function(tag1,n_breaks=5,xname="tag1",n_density=0,n_col=1){
hist(tag1, breaks=n_breaks,main = "Histogram of Outside
Temperature",density=n_density,col=n_col,xlab="Values")
}

PI Density Plot
This function calculates and plots the density of a select tag. The generic function density computes
kernel density estimates. Its default method does so with the given kernel and bandwidth for
univariate observations.

It lets you choose the color of the graphic as well as the ASP which is the ratio between axis y and x.

27

Figure 9 Graphic generated by R function PI_Density_Plot.

R
#Plotting density fundtion - It calculates and plots the density of numerical vector v
PI_Density_Plot <function(v,s_col="red",s_border="blue",s_type="l",n_asp=NULL,tagname="tag"){
d <- density(v[!is.na(v)])
plot(d, main="Distribution of Outside Temperature",type=s_type,asp=n_asp)
polygon(d, col=s_col, border=s_border)
}

PI Density Compare
This function is used if you want to compare the density between 2 or 3 tags. Outside and inside
temperature are compared. The package sm is required.

28

Figure 10 Graphic generated by R function PI_Density_Compare.

R
#Comparing density functions of two vectors ; can be extended to more than two vectors
PI_Density_Compare_TwoTags <- function(v,w,tag1name,tag2name){
require(sm)
sm.density.compare(c(v,w), c(rep(1, length(v)),rep(2, length(w))),xlab =
paste("Values"))
title(main="Comparison of Distributions")
legend("topright", c(1, 2), fill=2+0:1, legend=c("Outside Temperature", "Inside
Temperature"))
}

29

PI Box Plot

Produce box-and-whisker plot(s) of the given (grouped) values.

Figure 11 Graphic generated by R function PI_Box_Plot.

R
#Box plot of two tags compared - v and w are two numerical vectors
PI_Box_Plot <function(v,w,tag1name,tag2name,d_range=1.5,d_boxwex=0.8,d_staplewex=0.5){
boxplot(v,w,names=c("Outside Temperature","Inside
Temperature"),range=d_range,pars=list(boxwex=d_boxwex,staplewex=d_staplewex),main =
"Boxplot of Temperatures")
}

PI Regular Correlation
Correlations are useful because they can indicate a predictive relationship that can be exploited in
practice. For example, an electrical utility may produce less power on a warm and sunshine day based
on the correlation between electricity demand and weather.

30

Figure 12 Graphic generated by R function PI_Regular_Correlation.

R
#Regular Correlation between tags - v and w are two numerical vectors
PI_Regular_Correlation<- function(v,w,tag1name="tag1",tag2name="tag2"){
plot(v, w, xlab="Output Temparature", ylab="Total Power Demand", main="Scatter Plot")
}

One interesting feature to add on this function on your Windows Application is to show a Message
Box with the correlation coefficient before plotting the graphic. This coefficient varies between -1 and
1 and it is used as a measure of how correlated both vectors are.
PI Smooth Scatter
When some parts of the graphic generated using the PI Regular Correlation function is concentrated
with lots of dots, it is not possible to realize the level of concentration compared to other areas of the
graphic. PI Smooth Scatter function is a better option in this case. The levels of concentration of dots
are converted on tones of colors. The darker the color indicates a higher concentration of dots. This
function produces a smoothed color density representation of the scatterplot, obtained through a
kernel density estimate.

31

Figure 13 Graphic generated by R function PI_Smooth_Scatter.

R
#Smooth Scatter between tags

- v and w are two numerical vectors

PI_Smooth_Scatter<- function(v,w,tag1name="tag1",tag2name="tag2"){
smoothScatter(v, w, xlab="Output Temparature", ylab="Total Power Demand",main="Smooth
Scatter Plot")
}

PI Multi-Correlation
This function plots a graphic with multiple possible correlations between two tags in a group of three.
It requires the package gclus. The color on the background is related to the level of correlation
between the two tags, which could be measured using the correlation coefficient. The darker the
color, the more correlated are variables. With this function you can generate a big graphic showing all
possible correlations between five PI Points as shown on Figure 14.

32

Figure 14 Graphic generated by R function PI_Multi_Correlation3.

R
#Multi-Correlation between 3 tags
PI_Multi_Correlation3<- function(impact,tagnames){
tag<-tagnames
require(gclus)
library(gclus)
impact.r <- abs(cor(impact))
impact.col <- dmat.color(impact.r)
impact.o <- order.single(impact.r)
i=1
while (i<=length(impact.o))
{
tag[i]=tagnames[impact.o[i]]
i=i+1
}
cpairs(impact, impact.o, panel.colors=impact.col, gap=0.5, main="Color-coded
Correlation Matrix",labels=c(tag[1],tag[2],tag[3]))
}

33

R Source Function
This script calls the functions in the library. To call it I use the following command in R. Make sure the
paths are correct. Also notice that the \\ should be used instead of \ in the path because \ is the
escape character in the strings in R.

R
#This script incorporates the functions defined in Functions.R and is the one supposed
to be called and run directly
source("C:\\Program Files\\R\\R-2.15.1\\library\\Functions.R")

The code below should be executed on your application.


C#/R.NET
string rfilepath = @"C:\\Program Files\\R\\R-2.15.1\\library\\Functions.R";
engine.Evaluate("source(\"" + rfilepath + "\")");

34

7. GETTING DATA FROM THE PI SYSTEM


In this chapter there are some code snippets using some of our PI System Access technologies. Figure
15 shows which technologies could be used according to the chosen development platform.

PI AF SDK
R.NET

JRI

PI Web API

PI JDBC

AF & PI
Servers

PI Web API

Figure 15 PI System Access compatible with .NET Framework and Java.


There are examples for each PI Developer Technology for .NET Framework and Java about how to
retrieve data from PI to your application. On all the examples, the application will get compressed or
interpolated values from the PI archives. It is very important to provide the end user the option to to
retrieve interpolated values as many R functions, which were described on the previous chapter,
require vectors with the same size. Each vector has data from a single tag.
All the versions of the products of Figure 15 used are on the Overview section.

PI AF SDK
The PI AF SDK is a proprietary, low-level, object oriented library to program with AF. This library
exposes many classes that give you read and write access to various AF databases: Elements,
Attributes, AF Analysis Rules (ARs), and Data References (DRs). It also allows you to build AF
connectivity models and cases. Configuration of notification generators are done within the AF SDK.
The AF SDK library is written as native set of .NET assemblies. The .NET AF SDK library is installed in
the Global Assembly Cache (GAC) and is made of different assemblies:

OSIsoft.AF.Security.UI.dll;
OSIsoft.AF.UI.dll;
OSIsoft.AFSDK.dll.

Only the third library needs to be referenced on your project in case graphical user interface of the PI
AF SDK is not needed.

35

PI AF SDK cannot be used on Java environment.


In this case, PI AF SDK connects to the PI Server using OSIsoft.AF.PI namespace. The following code
snippet is an example using this library written in C#.

C#

private void GetData()


{
PIServers myPIServers= new PIServers();
PIServer MyPIServer = MyPIServers.DefaultPIServer;
MyTag = PIPoint.FindPIPoint(MyPIServer, "sinusoid");
string mode = "Recorded";
AFTime startTime = new AFTime("*-1d");
AFTime endTime = new AFTime("*");
AFTimeRange timeRange = new AFTimeRange(startTime, endTime);
AFTimeSpan timeSpan = AFTimeSpan.Parse("1h");
MyAtrbTag = AFAttribute.FindAttribute(@"\\" + MyPIServer.Name + @"\" + MyTag.Name,
null);
if (mode == "Recorded")
{
myValues = MyAtrbTag.Data.RecordedValues(timeRange,
OSIsoft.AF.Data.AFBoundaryType.Inside, null, null, false);
}
if (mode == "Interpolated")
{
myValues = MyAtrbTag.Data.InterpolatedValues(timeRange, timeSpan, null, null, false);
}
}

For more information please refer to the AF SDK Library Reference (file afsdk.chm), located on
%pihome%\HELP. This file is copied on this folder only if PI AF Developer Tools is installed on your
machine. The setup kit could be downloaded on the OSIsoft TechSupport Download Center.

PI JDBC

PI JDBC works in the same manner as the PI OLEDB Provider but offers an interaction with data
consumers (applications) that run on operating system that can be different than Windows.
PI JDBC uses Java API functions to send SQL statements to the PI Data Access Server (PI SQL DAS). The
PI SQL DAS implements a SQL engine allowing relational queries to be run against the PI System by
translating each query into a series of PI SDK calls. The PI SQL Data Access Server executes only under
36

the Windows operating system, meaning that an application built to be run under a Linux operating
system (it supports JDBC) would have to:
Send the query to the PI SQL Data Access Server using the Java PI API provided;
The PI SQL Data Access Server SQL engine would parse the query and translate it into PI SDK calls that
would then be transmitted to the PI Server (data store).
Two (2) layers must be cleared before the query arrives to the PI Server as opposed to only one for
the PI OLEDB provider. It also requires the PI SDK to allow the PI Data Access Server to function
properly. PI JDBC can only be used in Java and not in .NET environment.

PI WEB API

PI Web API provides basic functionality needed to retrieve and manipulate Time Series, Asset, and
Event Frame data. It is actually a RESTful web service on top of the PI System. There are a lot of content
about this product on PI DevClub:

White Paper - Developing with PI Web API


White Paper - Developing with PI Web API Indexed Search
PI Web API 2015 R3 - New features demo (Batch and Channels) - C#
How to use PI Web API with VBA - Introduction
Using PI Web API on HTML5 with jQuery
Using PI Web API on HTML5 with AngularJS
Optimizing web applications using PI Web API Batch
Using PI Web API with Python

There is an example using this product on the code snippet below.


C#
private void GetData()
{
if (mode == "Recorded")
{
PIPointRecData1Url = MyTag1.Links["Recorded Data"].Value + "?starttime=" + starttime +
"&endtime=" + endtime;
PIPointRecData = MakeRequest(PIPointRecData1Url).Result; }
if (mode == "Interpolated")
{
PIPointIntData1Url = MyTag1.Links["Interpolated Data"].Value + "?starttime=" +
starttime + "&endtime=" + endtime + "&interval=" + interval;
PIPointIntData = MakeRequest(PIPointIntData1Url).Result;}
}

private static async Task<dynamic> MakeRequest(string url)

37

{
WebRequest request = WebRequest.Create(url);
WebResponse response = request.GetResponse();
using (StreamReader sw = new StreamReader(response.GetResponseStream()))
{
using (JsonTextReader reader = new JsonTextReader(sw))
{
return JObject.ReadFrom(reader);
}
}
}

The code above can only be used on .NET Framework 4.5 as the asynchronous calls were added only
in that version.
For more information please refer to the PI Web API help accessing the page
https://webserver/piwebapi/help.
As PI Web API is a RESTful web service, it is compatible with the majority of frameworks including
Android SDK, iOS and probably PHP and Python. PI Web API is your best friend in case you are
developing an Android/iOS app which needs to retrieve data from the PI System.

38

8. INTEGRATING R AND OTHER SYSTEMS


This white paper has focused on the development of .NET Framework and Java application.
Nevertheless, it is known that there are other system/languages that could be integrated with the PI
System and R.
Table 4 shows the recommended R library and PI System Access technology to be used given a system
or a language as .NET Framework and Java.

System/Language

Applications

R library

PI System
Access
technology

Repository

Perl

WWW,
Interfaces

RSPerl

PI Web API

Omegahat

Python

WWW,
Interfaces

RSPython,
rpy

PI Web API

Sourceforge

Omegahat,

Java

Interfaces,
Graphics,WWW

rJava, JRI

PI JDBC

CRAN, RForge

Oracle, MySQL

Database

ROracle,
RmySQL

PI OLEDB
Enterprise

CRAN

.NET Framework

Interfaces,
Graphics, WWW

R.NET

PI AF SDK

Table 4 R library and PI System Access to be used according to the platform/language.

39

9. DEVELOPING APPLICATIONS
APPLICATION WITH GUI TO GENERATE GRAPHICS

There are three sample applications that generates R graphics with PI System data. All of them are
available on this folder from the white paper GitHub repository.
Windows Forms Application
One interesting Visual Studio project that you could be developed using R.NET is a Windows Form to
plot interesting graphics. Figure 16 shows us the dataflow of this type of project which is
unidirectional:

Figure 16 Dataflow for applications that generate graphics.

Firstly, data is sent to the application using PI AF SDK 2.6 (2014) which has Rich Data Access. This could
be done using the namespace OSIsoft.AF.PI, since it gives you the ability to connect directly to a PI
Server. Please refer to our webinar about PI AF SDK with RDA.
Once data is stored on PI AF SDK objects, it should be converted to R.NET objects and then be sent to
the R console, which is the feature responsible to plot the graphics.
There is a sample Windows Form application that you can download to use as a reference for
developing similar application. Here is what you should do:
1. Connect to the PI Server then choose the R function used to plot your graphic.
2. Type the PI Point names, start time and end time. The end user should choose between getting
data using Recorded Values or Interpolated Values.
3. If the option Interpolated Values is chosen, the end user should also define the interval.
4. With all of this information, a graphical will appear after the button to generate graphics is
pressed.
Figure 17 shows the Windows Form sample application.
40

Figure 17 Screenshot of the Windows Form Sample Application.

If you want to understand how this application works, please read the previous chapters about R
functions to generate graphics and about PI AF SDK, PI Web Services and PI Web API in order to
retrieve data from the PI System. You can also download the sample application to view its source
code.

Java Application

A very similar project could be developed on Java but using JRI + JavaGD instead of R.NET. PI JDBC, PI
Web Services or PI Web API should be used instead of PI AF SDK, since this library is not compatible to
Java. Figure 18 shows the Graphic User Interface of a similar application running on Java.

41

Figure 18 Screenshot of the Java GUI application.

Web Application

Web application and mobile apps are becoming more and more popular. This is why it is important to
be able to generate R graphics and display it on a web site or mobile app on the fly. If we try to use a
similar code from the Windows Form Application on a web site, it would pop up an R window on the
web server, instead of rendering an image on the client.
There is a web page that explains how to use ASP.NET with R.NET with a great sample on one of its
repositories. By referring to this example, a web application was developed and added to our GitHub
repository, retrieving data from the PI System and rendering SVG files to the web clients.

42

Figure 19 Screenshot of the web application.

DEVELOPING A PI ACE LIBRARY WITH R.NET


While the PI Asset Based Analytics does not provide a programmatic experience, it is possible to create
custom executables in PI ACE integrated with R.
Before starting a PI ACE project, you should be able to compile PI ACE calculations to target .NET
Framework 4, since this is required for using R.NET.
To change this setting, edit the PIACEClassLibraryHost.exe.config to:
XML
<?xml version="1.0"?>
<configuration>
<startup>
<supportedRuntime version="v4.0" />
</startup>
</configuration>

43

There
are
two
files
which
are
located
on
%pihome%\ACE\Scheduler
%pihome%\ACE\Scheduler\x86. Both of them should have the content above.

and

For more information, please refer to OSIsoft TechSupport KB00645.


PI ACE 2010 R2 introduces support for 64-bit calculations. You have the option to configure ACE to run
32 bit or 64 bit. The procedure is very similar of the previous setting, Please edit the file
PIACENetScheduler.exe.config located on %pihome%\ACE\Scheduler.
There are three possible options:
32 bits <add key="PIACEHostTargetCPU" value="x32"></add>
64 bits <add key="PIACEHostTargetCPU" value="x64"></add>
AnyCPU <add key="PIACEHostTargetCPU" value="AnyCPU"></add>
For more information, please refer to OSIsoft TechSupport KB00533.

Figure 20 Dataflow for PI ACE executables.

Below there are step-to-step instructions about how to create a PI ACE Project with R.NET that has
two input tags: sinusoid and cdt158 and three output tags Sd_sinusoid, Sd_Cdt158 and
Cor_cdt158_sinusoid.
On Visual Studio, create a new PI ACE projects selecting the input and output tags like a regular project.
Right click on the name of your project on your Solution Explorer window and select Properties.
Select the Compile ribbon and then click on the button Advanced Compile Options. A window
called Advanced Compiler Settings will appear. For the target framework, choose .NET Framework
4. On Target CPU, please select an option compatible with the value of the key PIACEHostTargetCPU
defined on the file PIACENetScheduler.exe.config.
The dataflow from PI to R is very similar to the Windows Application but in this case PI SDK and PI ACE
Libraries are used instead of PI AF SDK. If your final goal is to store the results in PI tags instead of
plotting some graphics, this option will address this objective.
44

On R, the standard deviation and correlation functions would be used and the results will be stored
on R variables.
The results are transferred back to PI ACE through R.NET.
Then, the R.NET objects are converted to PI SDK/PI ACE objects.
Finally, the values could be sent to the output tags using the method PutVal of the PI ACE library.

VB.NET
Dim sinusoid_val As NumericVector =
engine.CreateNumericVector(sinusoid_ArrayValues)
Dim cdt160_val As NumericVector =
engine.CreateNumericVector(cdt160_ArrayValues)
engine.SetSymbol("cdt160_val", cdt160_val)
engine.SetSymbol("sinusoid_val", sinusoid_val)
Dim Cor As NumericVector = engine.Evaluate("cor<-cor(sinusoid_val,
cdt160_val)").AsNumeric()
Dim Sd_cdt160 As NumericVector = engine.Evaluate("sd_cdt160<sd(cdt160_val)").AsNumeric()
Dim Sd_sinusoid As NumericVector = engine.Evaluate("sd_sinusoid<sd(sinusoid_val)").AsNumeric()

Above, there is an example about how to convert a double array to R.NET object using the SetSymbol
method. It is also shown how to use R function and send its results to your .NET application.

DEVELOPING CUSTOM INTERFACES


Another way of sending the R values back to PI is creating a new console application or a windows
service with PI AF SDK or any other PI System access technology, in a way that it behaves like an
interface. This could be developed on .NET Framework or on Java. There are two different scenarios.
Console Application to send random values from text files

For console applications you need to set some parameters in order to specify the hostname,
pointsource, location1, etc
When the application is started, it will connect to the PI Server which was specified on the hostname
parameter. Then, it will search for all tags with same point source and location1. For each one of the
found PI Points, the application will read the content of the file whose path was described on the
ExDesc point attribute, and create a vector on the memory of the application.

The frequency that the application sends data to PI is also defined on the parameter. Every cycle, the
application sends a value of the vector generated with the file (that contains the R code) defined of
the ExDesc of each tag. The next cycle will send the next value of its related vector.
45

The random values could be generated using the R code on the text files that would be ready by this
application. For instance:
R
x<-rnorm(1:100)

Console Application to use R functions


Another idea of using a Console Application with R.NET or JRI and a PI System Access technology is to
develop an application that behaves like an interface to use the R functions with PI Data and store the
results on PI Tags. This is similar of creating an ACE project.
You can use the location2 to define which R function you would like to use. The sourcetag point
attribute should store the source tag of your calculation. If there is more than 1 input, you could use
the ExDesc to define all tag names whose data are sent to R.
In order to develop these interfaces, you should have a good knowledge of .NET Framework (or Java)
and the chosen PI System Access method.

46

10. RETRIEVING PI DATA DIRECTLY INTO R


The previous chapters provide useful information in order to create application that communicates
with R and the PI System. Although there are a lot of use cases for the examples provided, there are
many request in order to retrieve PI data directly into R.
On this chapter, three alternatives are going to be shown to achieve this goal:

RClr
RCulr
RODBC

CREATING .NET LIBRARIES ON TOP OF PI AF SDK FOR


USING WIH RCLR
Our main PI Developer Technology is PI AF SDK developed on top of the .NET Framework. R has a
package called rClr which allows R to inter-operate with arbitrary .NET code. This means that using
this package, .NET libraries can be imported into R environment and their classes could be called
from R. Lets see how this works with PI AF SDK:

To install this package on R, just type on the R console:

R
install.packages("rClr",lib=.Library)

Use the library function to load any R package, including rClr:

R
library("rClr")

And load some R functions already used on this white paper:

R
source('C:\\Program Files\\R\\R-3.3.1\\library\\Functions.R');

Before writing any code, loading PI AF SDK library into R is needed using the clrLoadAssembly
function from rClr:

47

R
clrLoadAssembly("OSIsoft.AFSDK, Version=4.0.0.0, Culture=neutral,
PublicKeyToken=6238be57836698e6")

Now there are two approaches:

Import to R only the PI AF SDK and call its function directly using rClr.

Import to R not only the PI AF SDK but also a custom .NET library, which will be called by rClr.
This option is more interesting since writing .NET code should be easier than writing R code
using some rClr functions to call PI AF SDK. This way, rClr will instantiate an object of the
custom .NET library and call a method which will call all PI AF SDK methods under the hood.

As the second approach was chosen, a new Visual Studio Class Library project was created with
PISystemWrapper class as the main class. The content of this class is:

C#
public class PISystemWrapper
{
private double[] values1 = null;
private double[] values2 = null;
private double[] values3 = null;
public void GetPIData(string piDataArchiveName, string piPointName1, string
piPointName2, string piPointName3, string startTime, string endTime, string interval)
{
IEnumerable<string> piPointNames = new List<string> { piPointName1,
piPointName2, piPointName3 };
PIServer piServer = new PIServers()[piDataArchiveName];
if (piServer == null)
{
return;
}
IList<PIPoint> points = PIPoint.FindPIPoints(piServer, piPointNames);
PIPointList pointList = new PIPointList(points);
AFTimeRange timeRange = new AFTimeRange(new AFTime(startTime), new
AFTime(endTime));
AFTimeSpan timeSpan;
bool result = AFTimeSpan.TryParse(interval, out timeSpan);
if (result == false)
{
AFTimeSpan.TryParse("1h", out timeSpan);
}
IEnumerable<AFValues> valuesList = pointList.InterpolatedValues(timeRange,
timeSpan, string.Empty, false, new PIPagingConfiguration(PIPageType.TagCount, 100));

48

foreach(AFValues values in valuesList)


{
if (values.PIPoint.Name == piPointName1)
{
values1 = values.Where(m => m.ValueTypeCode == TypeCode.Single ||
m.ValueTypeCode == TypeCode.Double).Select(m => m.ValueAsDouble()).ToArray();
}
else if (values.PIPoint.Name == piPointName2)
{
values2 = values.Where(m => m.ValueTypeCode == TypeCode.Single ||
m.ValueTypeCode == TypeCode.Double).Select(m => m.ValueAsDouble()).ToArray();
}
else if (values.PIPoint.Name == piPointName3)
{
values3 = values.Where(m => m.ValueTypeCode == TypeCode.Single ||
m.ValueTypeCode == TypeCode.Double).Select(m => m.ValueAsDouble()).ToArray();
}
}
}

public double[] GetValues(int i)


{
if (i==1)
{
return values1;
}
else if (i == 2)
{
return values2;
}
else if (i == 3)
{
return values3;
}
else
{
return null;
}
}
}

Therefore, on R what you need to do in order to get data from the PI System with the
PISystemWrapper class is:

49

Import the custom .NET library


Create an instance of the PISystemWrapper object.
Call the GetPIData method with all the inputs, which will retrieve data from the PI
System using Bulk calls.
Call the GetValue method each time for each PI Point. This will return an array from
memory.

As a result, the R code below will retrieve data from the PI System through PISystemWrapper with
only some lines of code:

R
clrLoadAssembly('c:/1/rClrDotNetLibrary.dll')
w <- clrNew("rClrDotNetLibrary.PISystemWrapper")
clrCall(w, "GetPIData","marc-pi2016", "FAC.OAK.Weather-Inside_Humidity-Val.PV",
"FAC.OAK.Weather-Inside_Temperature-Val.PV", "FAC.OAK.Weather-Outside_TemperatureVal.PV", "1-Oct-2012", "1-Nov-2012", "1h");
v1=clrCall(w, "GetValues",as.integer(1))
v2=clrCall(w, "GetValues",as.integer(2))
v3=clrCall(w, "GetValues",as.integer(3))

If you take a look at the afsdk.R file on the GitHub repository, you will see a complete example of
retrieving PI data with rClr and PI AF SDK, and generating a graphic showing the correlations among
the streams.

MAKING RESTFUL WEBSERVICE CALLS AGAINST PI WEB


API DIRECTLY FROM R USING RCURL
As a result, the R code below will retrieve data from the PI System through PISystemWrapper with
only some lines of code:
The package rCurl is a wrapper for the libcurl library, which provides functions to allow making HTTP
requests and providing convenient functions to fetch URLs and process the results returned by the
web server. Therefore, rCurl is suitable for making HTTP requests against PI Web API.

On the previous section, it was shown how to get interpolated values using rClur and PI AF SDK. Lets
retrieve those same values using rCurl and PI Web API.

50

library("RCurl")
library("rjson")

getPIPointWebId <- function(piDataArchive, piPointName) {


path = paste(c("\\\\", piDataArchive, "\\" , piPointName))
compl_url= paste(c("/points?path=", path),collapse="");
x = getHttpRequest(compl_url)
x$WebId
}

getInterpolatedValues <- function(startTime, endTime, interval, webId1, webId2, webId3)


{
webIdsString= paste(c(webId1, webId2, webId3),collapse="&webId=");
print(webIdsString)
compl_url=
paste(c("/streamsets/interpolated?starttime=",startTime,"&endtime=",endTime,"&interval=
",interval,"&webId=", webIdsString),collapse="");
compl_url
x = getHttpRequest(compl_url)
}

getHttpRequest

<- function(compl_url) {

base_url = 'https://marc-web-sql.marc.net/piwebapi'
url = paste(c(base_url, compl_url),collapse="");
w=getURL(url,httpheader = c(Authorization = "Basic bsdgyYy5hZG06a2s="),
ssl.verifypeer = FALSE);
x=fromJSON(w);
}

tag1name = 'FAC.OAK.Weather-Inside_Humidity-Val.PV';
tag2name = 'FAC.OAK.Weather-Inside_Temperature-Val.PV';
tag3name = 'FAC.OAK.Weather-Outside_Temperature-Val.PV';
webId1 = getPIPointWebId('marc-pi2016',tag1name);
webId2 = getPIPointWebId('marc-pi2016',tag2name);
webId3 = getPIPointWebId('marc-pi2016',tag3name);
values = getInterpolatedValues('1-Oct-2012','1-Nov-2012','1h', webId1, webId2, webId3);

51

Some points about the code above:

The package rCurl is used to make the HTTP request and the package rJson is used to
convert the JSON string response into an R object.

All HTTP requests are made through the getHttpRequest method, which is called by the
getPIPointWebId and getInterpolatedValues.

Four HTTP GET requests are made. Three for getting the WebIds of the 3 PI Points. One
request for getting the interpolated values.

Using PI Web API Batch, it is possible to get the same results with one call only.
Nevertheless, it requires more time and effort in order to write the code.

This package allows you to edit the HTTP header on the getURL function. This is how the
Basic Authentication was added.

The ssl.verifypeer = FALSE was used due to the fact that the SSL certificate was self-signed. If
the SSL certificate is trusted by a trusted certificate authority, then this option wouldnt be
necessary.

RETRIEVING DATA FROM THE PI SYSTEM USING RODBC


AND PI ODBC.
A third alternative to retrieve PI data into R is using RODBC, which is an ODBC database interface. On
the other side, PI ODBC Driver is an ODBC 3.8 API-compliant driver that provides robust data access
to the PI System through the use of SQL queries. Therefore, RDOBC and PI ODBC can work together
in order to retrieve PI data.

Lets retrieve those same values using this new approach.

The first thing to do is to connect to the PI SQL DAS, which will call the PI Server through PI OLEDB
Provider or PI OLEDB Enterprise. In this case, PI OLEDB Enterprise is used.

R
dbhandle <- odbcDriverConnect('Driver={PI ODBC Driver};Server=MARC-WEB-SQL;Trusted
Connection=Yes;Provider Type=PIOLEDBENT;Initial Catalog=NuGreen; Provider String={Data
Source=MARC-PI2016; Integrated Security=SSPI};')

Note: PI OLEDB Enterprise 2016 is still read-only. Therefore, only use this product if you are sure that
you wont need to write/modify anything on the PI System.

Then, you just need to execute the SQL query and store the values on variables.

52

R
values <- sqlQuery(dbhandle, 'SELECT ea.Name As Name, Format(i.Time, \'MMM-dd-yyyy
HH:mm:ss\'), i.Value
FROM [IntegratingPISystemAndR].[Asset].[ElementHierarchy] eh
INNER JOIN [IntegratingPISystemAndR].[Asset].[ElementAttribute] ea
ON ea.ElementID = eh.ElementID,
[IntegratingPISystemAndR].[Data].[ft_InterpolateRange] i
WHERE i.ElementAttributeID = ea.ID
AND i.StartTime = \'1-Oct-2012\'
AND i.EndTime = \'1-Nov-2012\'
AND i.TimeStep = \'1h\'
OPTION (FORCE ORDER, EMBED ERRORS)')

tag1name = 'FAC.OAK.Weather-Inside_Humidity-Val.PV';
tag2name = 'FAC.OAK.Weather-Inside_Temperature-Val.PV';
tag3name = 'FAC.OAK.Weather-Outside_Temperature-Val.PV';

n = (dim(values)[1])/3;
v1 = 1: n;
v2 = 1: n;
v3 = 1: n;

i=1;
j=1;
k=1;
m = 1;

while(m<3*n) {

if (values[[1]][[m]]=="OutsideTemperature")
{
v3[i] = values[[3]][[m]]
i=i+1
}
if (values[[1]][[m]]=="Inside Temperature")
{
v2[j] = values[[3]][[m]]
j=j+1
}
if (values[[1]][[m]]=="Inside Humidity")
{
v1[k] = values[[3]][[m]]

53

k=k+1
}
m <- m+1;
}

Some comments about RODBC and PI ODBC:

PI OLEDB Enterprise 2016 is still read-only. Therefore, only use this product if you are sure
that you wont need to write/modify anything on the PI System.

PI ODBC can also be used with the PI Integrator for Business Analytics.

54

11. DEVELOPING THE R MULTICORRELATION PI CORESIGHT CUSTOM


SYMBOL

One hot topic nowadays within the OSIsoft community is PI Coresight extensibility, which allow users
to develop almost anything they want to. As long as you know about AngularJS, HTML5, JavaScript,
there is a lot you can do in order to add value to your business through those custom symbols.
There are a lot of customers who has asked some guidelines about integrating the PI System with R
due to the power of R shown on previous chapters. With PI Coresight extensibility, custom symbols
could be developed in order to show nice graphics generated with PI data. In this sample, for a given
amount of attributes, a graphic showing the correlation among those attributes will be displayed. As
a prerequisite, a custom ASP.NET Web API web service needs to be created in order to achieve this
goal.
Figure 21 shows how the final custom symbol looks like for 3 attributes.

Figure 21 R Multi-correlation PI Coresight custom symbol

DEVELOPING THE CUSTOM ASP.NET WEB API PROJECT


WITH R.NET
Let's start developing the custom ASP.NET Web API. By adding to this project PI AF SDK, Svg, R.NET
and R.NET Graphics, this RESTful web service can return SVG content from HTTP requests that the PI
Coresight custom symbol can consume.
55

On the R.NET GitHub, there is a sample code for using R.NET in ASP.NET, which was used for me to get
started. I also recommend you to take a look at this Visual Studio project. Although the
SvgGraphicsContext.cs and SvgGraphicsDevice.cs files were initially copied from this repository, some
small changes were made in order to work better.
Note that the RDotNet.Graphics NuGet package library is not released yet. The library is still an alpha
version that works pretty well. Concerning the Svg library, the most recent version does not work well
with RDotNet.Graphics library. Please use version 2.0.0.0 instead.
Let's start looking at the QueryData class, which is the input of our main action:
public class QueryData
{
public int Width { get; set; }
public int Height { get; set; }
public string StartTime { get; set; }
public string EndTime { get; set; }
public string Interval { get; set; }
public string[] Paths { get; set; }

public QueryData()
{
}
}

This means that the custom symbol will have to provide to our RESTful web service:
1)Width and Height of the symbol node
2)Time range that you want to analyze and generate the multi-correlation graphics
3)Paths from all the attributes
Let's see our controller and its unique action:
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Web.Http;
using RDotNet;
using RDotNet.Graphics;
using Svg;

56

using RMultWebService.Models;
using System.Web.Http.Cors;
using System.Web;
using System.Drawing.Imaging;
using System.Diagnostics;
using System.Configuration;

namespace RMultWebService.Controllers
{
public class CodeController : ApiController
{
private static REngine _engine = null;
private static SvgGraphicsDevice GraphicsDevice = null;
private static int lastWidth = -1;
private static readonly object _object = new object();

public CodeController()
{
if (_engine != null)
{
return;
}

_engine = REngine.GetInstance(null, true, null, null);


_engine.Initialize();
string rFilePath = ConfigurationManager.AppSettings["rFunctionPath"];
_engine.Evaluate("source(\"" + rFilePath + "\")");
GraphicsDevice = new SvgGraphicsDevice(new SvgContextMapper(400, 400,
SvgUnitType.Pixel, null));
_engine.Install(GraphicsDevice);

[HttpPost]
public IHttpActionResult Execute(QueryData queryData)
{
Stopwatch watch = new Stopwatch();
watch.Start();

try
{
if ((queryData.Paths == null) || (queryData.Paths.Count() < 2))

57

{
throw new Exception("There should be at least 2 attributes within
the symbol.");
}
if (queryData.Paths.All(m => m.Substring(0, 2) == "af") == false)
{
throw new Exception("PI Points are not accepted");
}
IEnumerable<string> plots = null;
RApplication app = new RApplication();
app.GetPIData(queryData.Paths, queryData.StartTime, queryData.EndTime,
queryData.Interval);

lock (_object)
{
System.Threading.Thread.Sleep(1000);
if (lastWidth != queryData.Width)
{
GraphicsDevice = new SvgGraphicsDevice(new
SvgContextMapper(queryData.Width, queryData.Height, SvgUnitType.Pixel, null));
_engine.Install(GraphicsDevice);
lastWidth = queryData.Width;
}

app.GenerateGraphic(_engine);
plots = GraphicsDevice.GetImages().Select(RenderSvg);
}
return Ok(plots);

}
catch (Exception ex)
{
return BadRequest(ex.Message + "|\n" + ex.Source + "|\n" +
ex.StackTrace);
}
finally
{
watch.Stop();
}
}

private static string RenderSvg(SvgDocument image)


{
using (var stream = new MemoryStream())
{

58

image.Write(stream);
stream.Position = 0;
using (var reader = new StreamReader(stream))
{
var contents = reader.ReadToEnd();
return contents;
}
}
}
}
}

The main action checks if QueryData.Paths is valid. This collection should have at least two items,
which should all be attributes (each path string must start with "af"). Those paths are provided by the
PI Coresight web services.
Then, an RApplication object is instantiated and the GetPIData() method retrieves PI Data.
The next step is to send the data to R and generate the graphics. Nevertheless, in order to avoid
exceptions, this code snippet should be written inside a lock(_object). This will make sure that there
won't be two requests running inside this piece of code simultaneously. Note that in order to generate
the Svg content, the GraphicsDevice needs to be "installed" on the REngine. If the width is changed
when compared to the last request, a new GraphicsDevice needs to be created and replaced.
The RApplication class was taken from Chapter 9 of the White Paper and it was simplified as only one
function will be used. The content of this class is shown below:
using System;
using System.Collections.Generic;
using System.Collections;
using System.Linq;
using System.Text;
using RDotNet;
using RDotNet.Graphics;
using System.IO;
using System.Diagnostics;
using OSIsoft.AF;
using OSIsoft.AF.Asset;
using OSIsoft.AF.Time;
using OSIsoft.AF.PI;
using System.Configuration;

59

namespace RMultWebService
{
public class RApplication
{

private PIValuesList piValuesList = null;


private readonly DateTime unixEpoch = new DateTime(1970, 1, 1, 0, 0, 0,
DateTimeKind.Utc);

public RApplication()
{
PISystems piSystems = new PISystems();
string piSystemName = ConfigurationManager.AppSettings["piSystemName"];
PISystem piSystem = piSystems[piSystemName];
}

public void GetPIData(string[] paths, string startTime, string endTime, string


interval)
{
string[] correctPaths = paths.Select(m => m.Substring(3)).ToArray();
AFKeyedResults<string, AFAttribute> results =
AFAttribute.FindAttributesByPath(correctPaths, null);
AFAttributeList attributeList = new AFAttributeList();
foreach (AFAttribute attribute in results)
{
attributeList.Add(attribute);
}

AFTime start = new AFTime(startTime);


AFTime end = new AFTime(endTime);
AFTimeRange timeRange = new AFTimeRange(start, end);
AFTimeSpan timeSpan = AFTimeSpan.Parse(interval);
IEnumerable<AFValues> valueResults =
attributeList.Data.InterpolatedValues(timeRange, timeSpan, string.Empty, false, new
PIPagingConfiguration(PIPageType.TagCount, 100));
piValuesList = new PIValuesList(valueResults);
}

public void GenerateGraphic(REngine engine)


{
for (int i = 1; i <= piValuesList.Count; i++)
{

double[] values = ConvertValuesToDoubleArray(piValuesList[i - 1]);

60

double[] ts = ConvertTSToDoubleArray(piValuesList[i - 1]);


NumericVector tagval = engine.CreateNumericVector(values);
engine.SetSymbol("tag" + i.ToString() + "val", tagval);
NumericVector tag_tsd = engine.CreateNumericVector(ts);
engine.SetSymbol("tag" + i.ToString() + "tsd", tag_tsd);
engine.Evaluate("tag" + i.ToString() + "ts<-as.POSIXct(tag" +
i.ToString() + "tsd, origin='1970-01-01')");
engine.Evaluate("tag" + i.ToString() + "<- data.frame(tag" +
i.ToString() + "ts,tag" + i.ToString() + "val)");
}
int[] arrayHelper = Enumerable.Range(1, piValuesList.Count).ToArray();
string[] tagnames = piValuesList.Select(m => "\"" + m.Name +
"\"").ToArray();
string impactString = string.Join(",", arrayHelper.Select(m => "tag" + m +
"$tag" + m + "val"));
string tnString = string.Join(",", tagnames);

engine.Evaluate("impact<-data.frame(" + impactString + ")");


engine.Evaluate("tn<-c(" + tnString + ")");
engine.Evaluate("PI_Multi_Correlation(impact,tn)");
}

private double GetUTCFormat(DateTime DateFormat)


{
double UtcFormat = 1;
TimeSpan ts = DateFormat - unixEpoch;
UtcFormat = (ts.TotalMilliseconds) / 1000;
return UtcFormat;

private double[] ConvertTSToDoubleArray(PIValues piValues)


{
return piValues.Select(val => GetUTCFormat(val.Timestamp)).ToArray();
}

private double[] ConvertValuesToDoubleArray(PIValues piValues)


{
return piValues.Select(val => Convert.ToDouble(val.Value)).ToArray();
}
}
}

61

Other classes such as PIValue.cs, PIValues.cs and PIValuesList.cs were also copied from Chapter 9.
Our custom web service to generate SVG content through R is ready!
According to weather you are using 32-bit or 64-bit R binaries, you might want to set the "Enable 32Bit Application" setting to True on IIS of the web server hosting your custom web service. Please refer
to Figure 22 for more details.

Figure 22 Enabling 32-bit applications on IIS

Let's move to the custom PI Coresight symbol.

62

DEVELOPING THE CUSTOM PI CORESIGHT SYMBOL


A great PI Coresight symbol starts with a great icon. Using public svg files for R, the r_multcorr.svg file
was generated. The icon on PI Coresight is shown on Figure 23:

Figure 23 R Multi-correlation PI Coresight custom symbol icon

On this chapter, the comments are about what is specific for this custom symbol. General PI Coresight
custom symbol development information can be found in many different resources, such as:

PI Coresight 2016 Extensibility Documentation


PI Coresight 3 Extensibility Samples and Tutorials
Developing the Google Maps PI Coresight symbol - Part 1

PI Coresight is developed on top of the AngularJS framework. The inject property of the definition
object makes some common AngularJS services available on the init function, including
the $http service for making RESTful web service calls. In this example. four services are injected:

$http this service is used for making a HTTP POST request against our custom web
service with R.NET and PI AF SDK.
$sce this service is used to show the content of the SVG on DOM.
$timeout this service is used to wait some seconds before running a function.
$document a wrapper for windows.document.

The links above will show more information directly on the AngularJS web site.
The code of the sym-rmult.js file is below:
(function (CS) {
var definition = {
typeName: 'rmult',
datasourceBehavior: CS.DatasourceBehaviors.Multiple,
iconUrl: 'Images/r-multcorr.svg',
inject: ['$http', '$sce', '$timeout', '$document'],
getDefaultConfig: function () {
return {
DataShape: 'Table',

63

Height: 400,
Width: 400,
Paths: null
};
},
init: init
};

function init(scope, elem, $http, $sce, $timeout, $document) {

var container = elem.find('#container')[0];


var id = "rmult_" + Math.random().toString(36).substr(2, 16);
container.id = id;
scope.id = id;

var timer;
scope.resize = function (width, height) {
$timeout.cancel(timer);
timer = $timeout(function () {
scope.config.Width = width;
scope.config.Height = height;
scope.showGraphic = false;
scope.updateGraphic();
}, 1000);
}

scope.updateGraphic = function () {

if (scope.config.ElementsList == undefined) {
return;
}

var postData = new Object();


postData.StartTime = "1-Oct-2012";
postData.EndTime = "1-Nov-2012";

64

postData.Interval = "1h";
postData.Width = scope.config.Width;
postData.Height = scope.config.Width;
postData.Paths = [];
for (var i = 0; i < scope.config.ElementsList.length; i++) {
postData.Paths.push(scope.config.ElementsList[i].Path);
}
if (postData.Paths.length < 2) {
alert('The symnol must have at least 2 attributes!');
return;
}
$http.post('http://marc-web-sql.marc.net:82/api/code',
postData).then(function (response) {
scope.plots = response.data;
scope.plot = scope.plots[0];
var currentElement = $document[0].getElementById(id)
var currentElementWrappedID = angular.element(currentElement);
currentElementWrappedID.height(scope.config.Width);
scope.config.Height = scope.config.Width;
scope.showGraphic = true;
})
}

scope.dataUpdate = function (data) {


if ((data == null) || (data.Rows.length == 0)) {
return;
}
if (data.Rows[0].Path) {
scope.config.ElementsList = data.Rows;
scope.updateGraphic();
}
}

scope.unsafe = function (s) {


return $sce.trustAsHtml(s);
}
return { dataUpdate: scope.dataUpdate, resize: scope.resize };
}
CS.symbolCatalog.register(definition);
})(window.Coresight);

65

The content of the sym_rmult_template.html is below:


<div id="container" style="width:100%;height:100%;">
<div ng-show="showGraphic==true" ng-bind-html="unsafe(plot)"
style="width:100%;height:100%;background: aliceblue;"></div>
<div ng-show="showGraphic==false" style="width:100%;height:100%;padding-top:40%;">
<img src="../../Coresight/Images/loading-large-gray.gif" />
</div>
</div>

Some comments about the code snippets above:

The initial lines of the init function has the goal of changing the id of the symbol in
order to make it unique.
Yes, $http can also be used for making HTTP RESTful web service calls against PI Web
API.
While the symbol is loading the SVG content from the web service, a gif image from PI
Coresight is displayed by using the ng-show directive and checking the content of
the scope.showGraphic variable.
R.NET Graphics generate better graphics if the width is the same value of the height.
The data object received on the dataUpdate function is used only to get the paths of
the attributes and not the values themselves. Those are retrieved by the custom web
service using the paths array string and the time range.
Only AF Attributes work on this example. If a PI Point is added to the symbol, an
exception will be throw and the response will have a status code of 500. But it is not
difficult to update the code and make PI Points also compatible.
The paths from the attributes are stored on the scope.config.ElementsList.

66

12. CONCLUSIONS
Integrating the PI System and R might be very valuable for you and your enterprise as you will be
taking advantage of the R functions and libraries to plot figures with PI data.
This white paper has focused on .NET Framework and Java development but there are other systems/
languages which are able to achieve the same goals. PI Web API, a RESTful web service on top of the
PI System is compatible with major languages from the market.
This white paper comes with some sample applications that you can refer to in order to develop your
own custom application.
For suggestions or enhancements related to this white paper, please send me an e-mail directly
(mloeff@osisoft.com).
Finally, in order for us to start a conversation, I would like to encourage you to post on PI Developers
Club, a description of the projects related to this topic that you want or have implemented on your
enterprise. This would also be the right channel if you are not able to follow one step of this white
paper.

Happy coding!!

67

REFERENCES
[1] Um Ambiente Grfico para Facilitar Tarefas de Data Mining via Ferramenta R, Joel Frederico
Azevedo Costa
[2]Tutorial JRI- Wojtek
[3] http://rosuda.org/R/JavaGD/
[4] http://en.wikipedia.org/wiki/R_(programming_language)
[5] http://rdotnet.codeplex.com/
[6] PI Application Development Course Version 1.3c
[7] http://www.wikihow.com/Install-the-Java-Software-Development-Kit
[8] http://www.vogella.com/tutorials/Eclipse/article.html
[9] http://rforge.net/JRI/

68

REVISION HISTORY
11-Dec-12

Initial framework by Ahmad Fattahi

27-May-14

First draft of the White Paper by Marcos Vainer Loeff

05-Jun-14

First version of the White Paper by Marcos Vainer Loeff

15-Jul-16

Second version of the White Paper by Marcos Vainer Loeff

69

You might also like