Big Data Databases

Big Data relies on a massive volume of both structured and unstructured data that is so large it is difficult to process using traditional database and software techniques. In most enterprise scenarios the volume of data is too big or it moves too fast or it exceeds current processing capacity.

2017 Open Source Big Data Databases

The databases and data warehouses you’ll find on these pages are the true workhorses of the Big Data world. They hold and help manage the vast reservoirs of structured and unstructured data that make it possible to mine for insight with Big Data. Businesses rely heavily on these open source solutions, from tools like Cassandra (originally developed by Facebook) to the well regarded MongoDB, which was designed to support the biggest of big data loads. And the tools rise to the challenge: OrientDB, for instance, can store up to 150,000 documents per second. It’s accurate to say that, as much as any tool set, the software listed on these pages plays a central role in today's global business marketplace.

Blazegraph

Formerly known as "Bigdata," Blazegraph is a highly scalable, high-performance database. It is available under an open source or a commercial license. Operating System: OS Independent.

Cassandra

Originally developed by Facebook, this NoSQL database is now managed by the Apache Foundation. It's used by many organizations with large, active datasets, including Netflix, Twitter, Urban Airship, Constant Contact, Reddit, Cisco and Digg. Commercial support and services are available through third-party vendors. Operating System: OS Independent.

CouchDB

Designed for the Web, CouchDB stores data in JSON documents that you can access via the Web or or query using JavaScript. It offers distributed scaling with fault-tolerant storage. Operating system: Windows, Linux, OS X, Android.

FlockDB

Best known as Twitter's database, FlockDB was designed to store social graphs (i.e., who is following whom and who is blocking whom). It offers horizontal scaling and very fast reads and writes. Operating System: OS Independent.

HBase

Another Apache project, HBase is the non-relational data store for Hadoop. Features include linear and modular scalability, strictly consistent reads and writes, automatic failover support and much more. Operating System: OS Independent.

Hibari

Used by many telecom companies, Hibari is a key-value, big data store with strong consistency, high availability and fast performance. Support is available through Gemini Mobile. Operating System: OS Independent.

Hive

Hadoop's data warehouse, Hive promises easy data summarization, ad-hoc queries and other analysis of big data. For queries, it uses a SQL-like language known as HiveQL. Operating System: OS Independent.

Hypertable

This NoSQL database offers efficiency and fast performance that result in cost savings versus similar databases. The code is 100 percent open source, but paid support is available. Operating System: Linux, OS X.

Infinispan

Infinispan from JBoss describes itself as an "extremely scalable, highly available data grid platform." Java-based, it was designed for multi-core architecture and provides distributed cache capabilities. Operating System: OS Independent.

InfoBright Community Edition

This scalable data warehouse supports data stores up to 50TB and offers "market-leading" data compression up to 40:1 for improved performance. Commercial products based on the same technology can be found at InfoBright.com. Operating System: Windows, Linux.

MongoDB

MongoDB was designed to support humongous databases. It's a NoSQL database with document-oriented storage, full index support, replication and high availability, and more. Commercial support is available through 10gen. Operating system: Windows, Linux, OS X, Solaris.

Neo4j

The "world’s leading graph database," Neo4j boasts performance improvements up to 1000x or more versus relational databases. Interested organizations can purchase advanced or enterprise versions from Neo Technology. Operating System: Windows, Linux.

OrientDB

This NoSQL database can store up to 150,000 documents per second and can load graphs in just milliseconds. It combines the flexibility of document databases with the power of graph databases, while supporting features such as ACID transactions, fast indexes

Redis

Sponsored by VMware, Redis offers an in-memory key-value store that can be saved to disk for persistence. It supports many of the most popular programming languages. Operating System: Linux.

Riak

Riak humbly claims to be "the most powerful open-source, distributed database you'll ever put into production." Users include Comcast, Yammer, Voxer, Boeing, SEOMoz, Joyent, Kiip.me, DotCloud, Formspring, the Danish Government and many others. Operating System: Linux, OS X.

Terrastore

Based on Terracotta, Terrastore boasts "advanced scalability and elasticity features without sacrificing consistency." It supports custom data partitioning, event processing, push-down predicates, range queries, map/reduce querying and processing and server-side update functions. Operating System: OS Independent.

More information: We hope this page was helpful and provided you with some information about big data databases. Check out our main page for more components of artificial intelligence resources.