Skip to content

Commit

Permalink
Merge branch 'datastore-client' of https://github.com/stfeng/YCSB int…
Browse files Browse the repository at this point in the history
…o datastore-client
  • Loading branch information
stfeng2 committed Oct 31, 2015
2 parents 855301b + 012e5bf commit 761926d
Show file tree
Hide file tree
Showing 18 changed files with 1,461 additions and 29 deletions.
5 changes: 5 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,9 @@ output*
.settings
.checkstyle

# ignore standard intellij
.idea/
*.iml
*.iws

.DS_Store
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,9 +32,9 @@ Getting Started
1. Download the latest release of YCSB:

```sh
curl -O --location https://github.com/brianfrankcooper/YCSB/releases/download/0.3.1/ycsb-0.3.1.tar.gz
tar xfvz ycsb-0.3.1.tar.gz
cd ycsb-0.3.1
curl -O --location https://github.com/brianfrankcooper/YCSB/releases/download/0.4.0/ycsb-0.4.0.tar.gz
tar xfvz ycsb-0.4.0.tar.gz
cd ycsb-0.4.0
```

2. Set up a database to benchmark. There is a README file under each binding
Expand Down
17 changes: 11 additions & 6 deletions bin/ycsb
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,7 @@ DATABASES = {
"cassandra-8" : "com.yahoo.ycsb.db.CassandraClient8",
"cassandra-10" : "com.yahoo.ycsb.db.CassandraClient10",
"cassandra-cql": "com.yahoo.ycsb.db.CassandraCQLClient",
"cassandra2-cql": "com.yahoo.ycsb.db.CassandraCQLClient",
"couchbase" : "com.yahoo.ycsb.db.CouchbaseClient",
"dynamodb" : "com.yahoo.ycsb.db.DynamoDBClient",
"elasticsearch": "com.yahoo.ycsb.db.ElasticSearchClient",
Expand All @@ -62,6 +63,7 @@ DATABASES = {
"infinispan-cs": "com.yahoo.ycsb.db.InfinispanRemoteClient",
"infinispan" : "com.yahoo.ycsb.db.InfinispanClient",
"jdbc" : "com.yahoo.ycsb.db.JdbcDBClient",
"kudu" : "com.yahoo.ycsb.db.KuduYCSBClient",
"mapkeeper" : "com.yahoo.ycsb.db.MapKeeperClient",
"mongodb" : "com.yahoo.ycsb.db.MongoDbClient",
"mongodb-async": "com.yahoo.ycsb.db.AsyncMongoDbClient",
Expand Down Expand Up @@ -138,11 +140,12 @@ def is_distribution():
# presumes maven can run, so should only be run on source checkouts
# will invoke the 'package' goal for the given binding in order to resolve intra-project deps
# presumes maven properly handles system-specific path separators
def get_classpath_from_maven(database):
# Given module is full module name eg. 'core' or 'couchbase-binding'
def get_classpath_from_maven(module):
try:
debug("Running 'mvn -pl com.yahoo.ycsb:"+database+"-binding -am package -DskipTests "
debug("Running 'mvn -pl com.yahoo.ycsb:" + module + " -am package -DskipTests "
"dependency:build-classpath -DincludeScope=compile -Dmdep.outputFilterFile=true'")
mvn_output = subprocess.check_output(["mvn", "-pl", "com.yahoo.ycsb:"+database+"-binding",
mvn_output = subprocess.check_output(["mvn", "-pl", "com.yahoo.ycsb:" + module,
"-am", "package", "-DskipTests",
"dependency:build-classpath",
"-DincludeScope=compile",
Expand Down Expand Up @@ -200,12 +203,14 @@ def main():
warn("Running against a source checkout. In order to get our runtime "
"dependencies we'll have to invoke Maven. Depending on the state "
"of your system, this may take ~30-45 seconds")
db_dir = os.path.join(ycsb_home, binding)
db_location = "core" if binding == "basic" else binding
project = "core" if binding == "basic" else binding + "-binding"
db_dir = os.path.join(ycsb_home, db_location)
# goes first so we can rely on side-effect of package
maven_says = get_classpath_from_maven(binding)
maven_says = get_classpath_from_maven(project)
# TODO when we have a version property, skip the glob
cp = find_jars(os.path.join(db_dir, "target"),
binding + "-binding*.jar")
project + "*.jar")
# alredy in jar:jar:jar form
cp.append(maven_says)
cp.insert(0, os.path.join(db_dir, "conf"))
Expand Down
65 changes: 65 additions & 0 deletions cassandra/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
<!--
Copyright (c) 2015 YCSB contributors. All rights reserved.
Licensed under the Apache License, Version 2.0 (the "License"); you
may not use this file except in compliance with the License. You
may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied. See the License for the specific language governing
permissions and limitations under the License. See accompanying
LICENSE file.
-->

# Cassandra (0.7, 0.8, 1.x) drivers for YCSB

**For Cassandra 2 CQL support, use the `cassandra2-cql` binding. The Thrift drivers below are deprecated, and the CQL driver here does not support Cassandra 2.1+.**

There are three drivers in the Cassandra binding:

* `cassandra-7`: Cassandra 0.7 Thrift binding.
* `cassandra-8`: Cassandra 0.8 Thrift binding.
* `cassandra-10`: Cassandra 1.0+ Thrift binding.
* `cassandra-cql`: Cassandra CQL binding, for Cassandra 1.x to 2.0. See `cassandra2/README.md` for details on parameters.

# `cassandra-10`

## Creating a table

Using `cassandra-cli`:

create keyspace usertable with placement_strategy = 'org.apache.cassandra.locator.SimpleStrategy' and strategy_options = {replication_factor:1};

create column family data with column_type = 'Standard' and comparator = 'UTF8Type';

**Note that `replication_factor` and consistency levels (below) will affect performance.**

## Configuration Parameters

- `hosts` (**required**)
- Cassandra nodes to connect to.
- No default.

* `port`
- Thrift port for communicating with Cassandra cluster.
* Default is `9160`.

- `cassandra.columnfamily`
- Column family name - must match the column family for the table created (see above).
- Default value is `data`

- `cassandra.username`
- `cassandra.password`
- Optional user name and password for authentication. See http://docs.datastax.com/en/cassandra/2.0/cassandra/security/security_config_native_authenticate_t.html for details.

* `cassandra.readconsistencylevel`
* `cassandra.scanconsistencylevel`
* `cassandra.writeconsistencylevel`

- Default value is `ONE`
- Consistency level for reads and writes, respectively. See the [DataStax documentation](http://docs.datastax.com/en/cassandra/2.0/cassandra/dml/dml_config_consistency_c.html) for details.
- *Note that the default setting does not provide durability in the face of node failure. Changing this setting will affect observed performance.* See also `replication_factor`, above.
19 changes: 3 additions & 16 deletions cassandra/src/main/java/com/yahoo/ycsb/db/CassandraCQLClient.java
Original file line number Diff line number Diff line change
Expand Up @@ -33,22 +33,9 @@

/**
* Tested with Cassandra 2.0, CQL client for YCSB framework
*
* In CQLSH, create keyspace and table. Something like:
* cqlsh> create keyspace ycsb
* WITH REPLICATION = {'class' : 'SimpleStrategy', 'replication_factor': 1 };
* cqlsh> create table usertable (
* y_id varchar primary key,
* field0 varchar,
* field1 varchar,
* field2 varchar,
* field3 varchar,
* field4 varchar,
* field5 varchar,
* field6 varchar,
* field7 varchar,
* field8 varchar,
* field9 varchar);
*
* See {@code cassandra2} for a version compatible with Cassandra 2.1+.
* See {@code cassandra2/README.md} for details.
*
* @author cmatser
*/
Expand Down
73 changes: 73 additions & 0 deletions cassandra2/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
<!--
Copyright (c) 2015 YCSB contributors. All rights reserved.
Licensed under the Apache License, Version 2.0 (the "License"); you
may not use this file except in compliance with the License. You
may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied. See the License for the specific language governing
permissions and limitations under the License. See accompanying
LICENSE file.
-->

# Apache Cassandra 2.x CQL binding

Binding for [Apache Cassandra](http://cassandra.apache.org), using the CQL API
via the [DataStax
driver](http://docs.datastax.com/en/developer/java-driver/2.1/java-driver/whatsNew2.html).

To run against the (deprecated) Cassandra Thrift API, use the `cassandra-10` binding.

## Creating a table for use with YCSB

For keyspace `ycsb`, table `usertable`:

cqlsh> create keyspace ycsb
WITH REPLICATION = {'class' : 'SimpleStrategy', 'replication_factor': 3 };
cqlsh> USE ycsb;
cqlsh> create table usertable (
y_id varchar primary key,
field0 varchar,
field1 varchar,
field2 varchar,
field3 varchar,
field4 varchar,
field5 varchar,
field6 varchar,
field7 varchar,
field8 varchar,
field9 varchar);

**Note that `replication_factor` and consistency levels (below) will affect performance.**

## Cassandra Configuration Parameters

- `hosts` (**required**)
- Cassandra nodes to connect to.
- No default.

* `port`
* CQL port for communicating with Cassandra cluster.
* Default is `9042`.

- `cassandra.keyspace`
Keyspace name - must match the keyspace for the table created (see above).
See http://docs.datastax.com/en/cql/3.1/cql/cql_reference/create_keyspace_r.html for details.

- Default value is `ycsb`

- `cassandra.username`
- `cassandra.password`
- Optional user name and password for authentication. See http://docs.datastax.com/en/cassandra/2.0/cassandra/security/security_config_native_authenticate_t.html for details.

* `cassandra.readconsistencylevel`
* `cassandra.writeconsistencylevel`

* Default value is `ONE`
- Consistency level for reads and writes, respectively. See the [DataStax documentation](http://docs.datastax.com/en/cassandra/2.0/cassandra/dml/dml_config_consistency_c.html) for details.
* *Note that the default setting does not provide durability in the face of node failure. Changing this setting will affect observed performance.* See also `replication_factor`, above.
61 changes: 61 additions & 0 deletions cassandra2/pom.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
<?xml version="1.0" encoding="UTF-8"?>

<!--
Copyright (c) 2012-2015 YCSB contributors. All rights reserved.
Licensed under the Apache License, Version 2.0 (the "License"); you
may not use this file except in compliance with the License. You
may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied. See the License for the specific language governing
permissions and limitations under the License. See accompanying
LICENSE file.
-->

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>com.yahoo.ycsb</groupId>
<artifactId>binding-parent</artifactId>
<version>0.5.0-SNAPSHOT</version>
<relativePath>../binding-parent</relativePath>
</parent>

<artifactId>cassandra2-binding</artifactId>
<name>Cassandra 2.1+ DB Binding</name>
<packaging>jar</packaging>

<dependencies>
<!-- CQL driver -->
<dependency>
<groupId>com.datastax.cassandra</groupId>
<artifactId>cassandra-driver-core</artifactId>
<version>${cassandra2.cql.version}</version>
</dependency>
<dependency>
<groupId>com.yahoo.ycsb</groupId>
<artifactId>core</artifactId>
<version>${project.version}</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.cassandraunit</groupId>
<artifactId>cassandra-unit-shaded</artifactId>
<version>2.1.9.2</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.12</version>
<scope>test</scope>
</dependency>

</dependencies>
</project>
Loading

0 comments on commit 761926d

Please sign in to comment.