Oracle NoSQL Database

Virtual Developer Day –
Oracle NoSQL Database

Watch presentations and demonstrations
and take part in hands-on labs moderated by
Oracle experts.
Brought to you by Oracle Technology Network
Presenter: Dave Segleau

Title: Director Product Management
1 | © 2011 Oracle Corporation –
Proprietary and Confidential
The following is intended to outline our general product
direction. It is intended for information purposes only, and may
not be incorporated into any contract. It is not a commitment to
deliver any material, code, or functionality, and should not be
relied upon in making purchasing decisions. The development,
release, and timing of any features or functionality described
for Oracle s products remains at the sole discretion of Oracle.
Virtual Developer Day – Oracle NoSQL Database 2 | © 2011 Oracle Corporation –
Brought to you by Oracle Technology Network Proprietary and Confidential

Agenda
•  Big Data and NoSQL

•  Oracle NoSQL DB Feature Overview
•  NoSQL DB API Overview
•  NoSQL DB Application Code Walkthrough

Big Data Lifecycle
DECIDE ACQUIRE
Make
Be'er
Decisions
Using
ANALYZE ORGANIZE
Big Data

Oracle Big Data Platform
Big Data
Appliance Exadata Exalytics
Hadoop
Oracle
In-‐Database AnalyBcs
Advanced
Analytic
Analytics Applications!
Open Source R Oracle Big Data Alerts,
Connectors Data
Dashboards, MD-‐
Warehouse
Oracle NoSQL InfiniBand InfiniBand Analysis, Reports,
Database Query
Oracle Data Oracle
Integrator Database Web Services
Applications BI AbstracBon
!
ACQUIRE ORGANIZE ANALYZE DECIDE

Big Data Acquisition Characteristics
Batch-‐Oriented Real-‐Time
Process data to use Deliver a service
Bulk storage Fast access to speciﬁc record
Write once, read all Read, write, delete update

Big Data Storage Choices
Hadoop Distributed File
System (HDFS)
File System Database
Parallel scanning Indexed storage

No inherent structure Simple data structure
High volume writes High volume random reads and writes
Batch Oriented Real-‐Time

Agenda

•  Oracle NoSQL Feature Overview

A Distributed, Scalable Key-Value Database
Application Application
Simple Data Model
NoSQL Database NoSQL Database
Driver Driver
Small, distributed footprint

Highly scalable, available

Transparent load balancing

Integrates with Oracle Stack
Storage Nodes Storage Nodes
Datacenter A Datacenter B

Architecture Summary
Scalable, Highly Available, Optimized
•  Scalability
–  Dynamic data partitioning and distribution
–  Optimized data access via intelligent driver
•  High availability
–  One or more replicas
–  Resilient to partition master failures
–  No single point of failure
–  Disaster recovery through location of replicas
•  Transparent load balancing
–  Reads from master or replicas
–  Driver is network topology & latency aware

Oracle NoSQL Database Use Cases
Use Cases
Customer profile management
Social networks
Personalization
SIMPLE QUERIES
Click-through data processing

High-throughput event processing
DYNAMIC SCHEMA
Sensor & statistics data capture

Authentication & Content management
HIGH VOLUME DATA
Mobile application backend infrastructure
Archiving

Simple Data Model
Key-value pairs
•  Simple data model – key-value pair (major+minor-key paradigm)
•  Simple operations – read/insert/update/delete, RMW support
•  Scope of transaction – records within a major key, single API call
•  Unordered scan of all data (non-transactional)
Major key: userid
Strings
Minor key: subscriptions address
Byte Array à Value: expiration date phone # email id

Simple Data Model
ACID Transactions
•  ACID transactions by default
•  Transaction Scope
–  Single API call
–  All records must have the same major key
–  Support for multiple operations within a transaction
•  Can be relaxed for increased performance on a per-

operation basis

Simple Data Model
ACID Transactions – Configurability
•  Configurable Durability Policy
•  Configurable Consistency Policy

Oracle NoSQL Database Differentiation
Integrates seamlessly with Oracle Stack (ODI, CEP, OLH)
Commercial Grade Scalability and Simple

Software and Support Availability Simple Data Model Administration
•  General Purpose •  Intelligent Oracle •  Simple Major + •  Web-based Console

NoSQL DB Driver Minor key and Value and CLI commands
•  Reliable – Based •  Evenly distributes Data data structure
on proven Berkeley •  Sends operation to •  Manages and Monitors:
DB JE HA fastest node •  ACID transactions •  Topology
•  Bounded network hops •  Load
for all operations •  Performance
•  Easy to Install & •  Configurable
Configure consistency and •  Events
•  Automatic replication •  Alerts
durability
and failover

Benchmarking
• YCSB Benchmark
• 1.6 billion records
• 94K insert/sec
• 25K read/update/sec
• Low latency
• Linear scalability

ü Easy to use, easy to manage
ü Scalable, Available, Predictable Latency
ü A NoSQL Database from a vendor you trust

Agenda


Getting Started with Oracle NoSQL DB
1.  Download from OTN:

www.oracle.com/technetwork/products/nosqldb/downloads/index.html
2.  Review Quick Start & Getting Started Guide
3.  Review Programmatic API Guide
4.  Install NoSQL DB
1.  kvlite or
2.  full NoSQL DB installation
5.  Write Java code

Single Node Installation – Introducing KVLite
•  Single-node database without replication

•  Used to develop and test access to Oracle NoSQL
Database
•  Easy to get started
•  Included in NoSQL DB download
•  Not intended for production

Starting kvlite
•  Requires Java 1.6.0.u25 or later

java –version
•  Starting kvlite
cd kv-‐1.2.123/lib
java -‐jar kvstore-‐1.2.123.jar kvlite -‐store dbStore –admin
<portID> -‐port <portID> -‐host localhost –root ../kvroot

•  Is kvlite running?
java -‐jar kvstore-‐1.2.123.jar ping -‐port <portID> -‐host
localhost

API Overview – Open and Close
•  Uses KVStore handle

•  Open a connection
KVStoreFactory.getStore(new(KVStoreConfig))
•  Close a connection & release resources
Close()
•  KVStore handle used to create, read, insert, update and
delete records (key-value pairs)


API Overview – Configuration
•  Uses KVStoreConfig object

–  Specifies store name
–  Specifies at least one helper host
–  Provides methods to set and get policy for
•  Consistency
•  Durability
•  Request management
KVStoreConfig(“dbStore”, “localhost:5000”)

API Overview – Keys and Values
•  Create a Key
•  Major and Minor Paths
Key id = Key.createKey(“sue.smith@Example.com”);
•  Create a Value
String name = “Susan Smith”;
Value vName = Value.createValue(name.getBytes());
•  Exceptions to Handle during Key-value operations
•  DurabilityException
•  RequestTimeoutException
•  FaultException

API Overview – Record Versions
•  Specific version of a record (key-value pair)

•  Returned on put() and get() operations
•  Version is just a token, not a sequence number
•  Used to Serialize an update after a read operation
putifVersion(Key, Value, Version);

API Overview – Inserting & Updating Data
•  Version = kvstore.put(Key, Value);

•  Insert or Update key/value pair
•  Returns Version
•  Version = kvstore.putIfAbsent(Key, Value);
•  Insert a new key/value pair
•  If key already exists, return a null Version
•  Version = kvstore.putIfPresent(Key, Value);
•  Update an existing key/value pair
•  If key does not exist, return a null Version
•  Version = kvstore.putIfVersion(Key, Value, Version);
•  Update an existing key/value pair
•  If key does not exist or version is different, return a null Version; else return new Version

API Overview – Deleting Data
kvstore.delete()
Kvstore.deleteIfVersion(Key, Version)
•  Delete the key/value pair associated with the key

•  Specifying the record Version is optional

API Overview – Reading Data
ValueVersion = kvstore.get(Key);
ValueVersion = kvstore.get(Key, Consistency, Timeout,
TimeoutUnit);
•  Returns the ValueVersion associated with the supplied key

•  Consistency, Timeout, TimeoutUnit are optional
•  Default Consistency is None
getValue(ValueVersion) will return Value

getVersion(ValueVersion) will return Version

API Overview – Multi-operation Transactions
•  Transaction scope == a single command

•  Supported operations: put() & delete()
•  All keys must share the same major path
•  Create list of operations using OperationFactory Interface
final OperationFactory factory = dbStore.getStore
().getOperationFactory();
final List<Operation> ops = new ArrayList<Operation>();
ops.add(factory.createPut(Key, Value));

Virtual Developer Day – Oracle NoSQL Database
Brought to you by Oracle Technology Network
29 | © 2011 Oracle Corporation –
Proprietary and Confidential
API Overview – Multi-record Operations
•  Efficient since not all records brought in memory at once.

kvstore.multiDelete(Key, KeyRange, Depth)
kvstore.multiGet(Key, KeyRange, Depth)
kvstore.multiGetIterator(Direction, BatchSize, Key, KeyRange, Depth)
kvstore.multiGetKeys(Key, KeyRange, Depth)
kvstore.multiGetKeysIterator(Direction, BatchSize, Key, KeyRange, Depth)
kvstore.storeIterator(Direction, BatchSize, Key, KeyRange, Depth))
•  KeyRange contains the start/stop key values
•  Depth: CHILDREN_ONLY , DESCENDANTS_ONLY, PARENT_AND_CHILDREN ,
PARENT_AND_DESCENDANTS
•  Direction: FORWARD, REVERSE, UNORDERED

API Overview – Write Durability
•  Controlled via a Durability object, specifying

–  Master write durability,
–  Replica write durability
–  Replication acknowledgement policy
Durability durability = new Durability(
(Durability.SyncPolicy.NO_SYNC,
Durability.SyncPolicy.NO_SYNC,
Durability.ReplicaAckPolicy.SIMPLE_MAJORITY);
kvstore.put(Key, Value, null, durability, 0, null);

API Overview – Read Consistency
•  Controlled via a consistency specification, specifying

–  Read consistency level,
–  Timeout,
–  Timeout time units
ValueVersion = kvstore.get(Key, Consistency.NONE, 10,
TimeUnit.MILLISECONDS);

API Overview – Summary
•  Open/Close connections
getstore(), close()
•  Single-record operations
get…(), put…(), delete…()
•  Multi-operation transactions
OperationFactory, ops.add()
•  Multi-record operations
multiget…(), multidelete()
•  Operational support
–  Record Versions, Durability Policy, Consistency Policy

Oracle NoSQL DB Resources
Documentation
•  On OTN and in download
–  docs.oracle.com/cd/NOSQL/html/index.html
•  Getting Started Guides
•  Programmatic API
•  Installation & Release Notes
•  FAQ

Oracle NoSQL DB Resources
Support & Technical Information
•  Support via OTN forums and Oracle Support process
•  OTN Forum:
–  Forum Home » Big Data » NoSQL Database
–  forums.oracle.com/forums/forum.jspa?forumID=1388
•  Oracle.com landing page:
–  www.oracle.com/us/products/database/nosql/overview/index.html
•  OTN (including documentation and download):
–  www.oracle.com/technetwork/products/nosqldb/overview/
index.html

Agenda


Where to find sample code
•  NoSQL DB examples directory

–  Hadoop: Reads that from NoSQL DB using Hadoop/MapReduce
–  Hello: Simple application that reads/writes a single record
–  Schema: Application that reads/writes multiple records, illustrates
basic schema design principals and operations , including:
•  Schema definition
•  Key and Value creation
•  Insert, Update, Delete
•  Lookup and Iteration

NoSQL DB Data Loader Application
•  Generic application that loads data into a NoSQL Database

–  HDFS and non-HDFS versions
–  Simple delimiter formatted or XML input files
–  Show all keys option
–  Delete all keys option
•  Source code can be found on NoSQL DB documentation page on
OTN:
www.oracle.com/technetwork/products/nosqldb/documentation/index.html

What we’re showing
•  Create a Database connection

KVStoreFactory.getStore(new(KVStoreConfig))
•  Iterate through all of the keys
–  Delete keys if requested
–  Show keys if requested
kvstore.storeKeysIterator()
key.getFullPath()
kvstore.delete()
•  Writing data
kvstore.put()

ü Easy to use, easy to manage
ü Scalable, Available, Predictable Latency
ü A NoSQL Database from a vendor you trust

Join the Oracle NoSQL DB Community
h'p://www.oracle.com/us/technologies/big-‐data/index.html
OTN Forum:
forums.oracle.com -‐> Big Data -‐> NoSQL

Charles Lamb’s blog:
Blogs.oracle.com/charleslamb/

Developer Webcast Series

Virtual Developer
Day – Oracle NoSQL Database 41 | © 2011 Oracle Corporation –

APPENDIX

Oracle NoSQL DB Licensing
Community VS Enterprise Edition
•  Two versions
–  Oracle NoSQL Database Community Edition. Open Source. AGPL
license.
–  Oracle NoSQL Database Enterprise Edition. Closed Source. Standard
Oracle License.
•  Community Edition has all of the basic functionality and APIs. Gets
you started. Competes with other OS NoSQL solutions.
•  Enterprise Edition for large, production, multi-data center, Oracle

integration centric customers and/or non-GPL compliant customers.

Benchmarking Configurations
•  YCSB-based benchmark (Yahoo Cloud Services Benchmark)

–  Key ~13 bytes, Data ~= 1.1K
•  Configurations of 3 (1x3) – 192 (64 x 3) storage nodes
–  Replication factor of 3 (master + 2 replicas)
–  100m to 2.1b records, 100m-400m records per storage node
–  Intel Systems: 2.93ghz Intel Westmere (wds024c) model x5670, dual
socket with 6 cores/socket, 24GB of memory, single 300GB local disk
and RedHat 2.6.18-164.11.1.el5.crt1
–  Cisco Systems: UCS C200 M2 & UCS C210 M2 systems (Intel 5600s),
dual socket with 6 cores/socket, 18GB of memory, 4,8 or 16 disks for
total of 8-16TB.

Benchmarking Configurations
Systems configured to minimize I/O overhead
•  Btree fits in memory è one I/O per record read
•  Writes are buffered + log structured storage system è
fast write throughput
•  GC and File System tuning to optimize throughput


Oracle NoSQL Database

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Oracle NoSQL Database

Uploaded by

Copyright:

Available Formats

Virtual Developer Day –

Oracle NoSQL Database

Brought to you by Oracle Technology Network

Presenter: Dave Segleau

Virtual Developer Day – Oracle NoSQL Database 2 | © 2011 Oracle Corporation –

Brought to you by Oracle Technology Network Proprietary and Confidential

• Big Data and NoSQL

Virtual Developer Day – Oracle NoSQL Database 3 | © 2011 Oracle Corporation –

Brought to you by Oracle Technology Network Proprietary and Confidential

Virtual Developer Day – Oracle NoSQL Database 4 | © 2011 Oracle Corporation –

Brought to you by Oracle Technology Network Proprietary and Confidential

ACQUIRE ORGANIZE ANALYZE DECIDE

Virtual Developer Day – Oracle NoSQL Database 5 | © 2011 Oracle Corporation –

Brought to you by Oracle Technology Network Proprietary and Confidential

Process data to use Deliver a service

Bulk storage Fast access to speciﬁc record

Write once, read all Read, write, delete update

Virtual Developer Day – Oracle NoSQL Database 6 | © 2011 Oracle Corporation –

Brought to you by Oracle Technology Network Proprietary and Confidential

Parallel scanning Indexed storage

Brought to you by Oracle Technology Network Proprietary and Confidential

• Big Data and NoSQL

Virtual Developer Day – Oracle NoSQL Database 8 | © 2011 Oracle Corporation –

Brought to you by Oracle Technology Network Proprietary and Confidential

Virtual Developer Day – Oracle NoSQL Database 9 | © 2011 Oracle Corporation –

Brought to you by Oracle Technology Network Proprietary and Confidential

Virtual Developer Day – Oracle NoSQL Database 10 | © 2011 Oracle Corporation –

Brought to you by Oracle Technology Network Proprietary and Confidential

Virtual Developer Day – Oracle NoSQL Database 11 | © 2011 Oracle Corporation –

Brought to you by Oracle Technology Network Proprietary and Confidential

Byte Array à Value: expiration date phone # email id

Virtual Developer Day – Oracle NoSQL Database 12 | © 2011 Oracle Corporation –

Brought to you by Oracle Technology Network Proprietary and Confidential

• Can be relaxed for increased performance on a per-

Brought to you by Oracle Technology Network Proprietary and Confidential

• Configurable Consistency Policy

Virtual Developer Day – Oracle NoSQL Database 14 | © 2011 Oracle Corporation –

Brought to you by Oracle Technology Network Proprietary and Confidential

Commercial Grade Scalability and Simple

• General Purpose • Intelligent Oracle • Simple Major + • Web-based Console

Virtual Developer Day – Oracle NoSQL Database 15 | © 2011 Oracle Corporation –

Brought to you by Oracle Technology Network Proprietary and Confidential

• 1.6 billion records

Virtual Developer Day – Oracle NoSQL Database 16 | © 2011 Oracle Corporation –

Brought to you by Oracle Technology Network Proprietary and Confidential

ü Easy to use, easy to manage

ü Scalable, Available, Predictable Latency

ü A NoSQL Database from a vendor you trust

Virtual Developer Day – Oracle NoSQL Database 17 | © 2011 Oracle Corporation –

Brought to you by Oracle Technology Network Proprietary and Confidential

• Big Data and NoSQL

Virtual Developer Day – Oracle NoSQL Database 18 | © 2011 Oracle Corporation –

Brought to you by Oracle Technology Network Proprietary and Confidential

1. Download from OTN:

Brought to you by Oracle Technology Network Proprietary and Confidential

• Single-node database without replication

Virtual Developer Day – Oracle NoSQL Database 20 | © 2011 Oracle Corporation –

Brought to you by Oracle Technology Network Proprietary and Confidential

• Requires Java 1.6.0.u25 or later

Brought to you by Oracle Technology Network Proprietary and Confidential

• Uses KVStore handle

Brought to you by Oracle Technology Network Proprietary and Confidential

• Uses KVStoreConfig object

Virtual Developer Day – Oracle NoSQL Database 23 | © 2011 Oracle Corporation –

•  Big Data and NoSQL

•  Big Data and NoSQL

•  Can be relaxed for increased performance on a per-

•  Configurable Consistency Policy

•  General Purpose •  Intelligent Oracle •  Simple Major + •  Web-based Console

• 1.6 billion records

ü Easy to use, easy to manage

ü Scalable, Available, Predictable Latency

ü A NoSQL Database from a vendor you trust

•  Big Data and NoSQL

1.  Download from OTN:

•  Single-node database without replication

•  Requires Java 1.6.0.u25 or later

•  Uses KVStore handle

•  Uses KVStoreConfig object

•  Specific version of a record (key-value pair)

•  Version = kvstore.put(Key, Value);

•  Delete the key/value pair associated with the key

•  Returns the ValueVersion associated with the supplied key

•  Transaction scope == a single command

•  Efficient since not all records brought in memory at once.

•  Controlled via a Durability object, specifying

•  Controlled via a consistency specification, specifying

•  Big Data and NoSQL

•  NoSQL DB examples directory

•  Generic application that loads data into a NoSQL Database

•  Create a Database connection

ü Easy to use, easy to manage

ü Scalable, Available, Predictable Latency

ü A NoSQL Database from a vendor you trust

•  Enterprise Edition for large, production, multi-data center, Oracle

•  YCSB-based benchmark (Yahoo Cloud Services Benchmark)