Storage

Out of the box, the following storage technologies/plugins are supported.

PlainStoragePlugin

This plugin serializes samples using either Google’s ProtocolBuffers (PB) or Apache Parquet and stores data in chunks. Each chunk has a well defined key and stores data for one PV for a well defined time duration (for example, a month). Using Java NIO2, one can store each chunk in 1. A file per chunk resulting in a file per PV per time partition. 2. A zip file entry in a .zip file per chunk resulting in a .zip file per PV. 3. This can be extended to use other storage technologies for which a NIO2 provider is available (for example, Amazon S3, a database BLOB per chunk or a key/value pair per chunk in any key/value store).

Note

By default, the PlainStoragePlugin maps PV names to keys using a simple algorithm that relies on the presence of a good PV naming convention. To use your own mapping scheme, see the Key Mapping section in the customization guide.

To add support for other storage technologies - see the customization guide for details.