Skip to main content

Cloud Storage

DEFAULT RETURNERWhich cloud returner should be used as default? Only change this value if you know what you are doing.strcloud-



MAX BATCH SIZEThe maximum amount of entries to include in an upload batch.int1000-
MAX RETRYThe maximum retry attempts before transferring batch to fail queue. Retry and fail queues are disabled if set to zero, meaning that pending will block until success.int30-
FAIL TTLHow many days of retention before a fail queue is deleted?int30d


LEVELInteger from 0 to 9 controlling the level of compression; 1 is fastest and produces the least compression, and 9 is slowest and produces the most compression. 0 is no compression.int9-
ALGORITHMCompress data with selected algorithm before uploading.strNone-


URLThe endpoint URL of the API where data is pushed to.url
AUTH TOKENThe authentication token used in HTTP header when pushing data to the API.strXXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX-


SPLAYThe maximum amount of time in seconds between consecutive uploads of batches. The actual delay is a randomly generated value between zero and the splay value. In error scenarios, the delay is increased by an incrementing factor.int10s
INTERVALHow often should cached data be uploaded to the cloud endpoint?float60s
TIMEOUTHow long to wait for server to respond?int60s
EVENT REGEXIn addition to repeated interval, upload data to cloud when event(s) occur.str^system/power/(sleep|hibernate|reboot)-

Jsonl Returner#

FILENAME FORMATHow should the JSONL files be named?str{now:%y%m%d%H%M%S}_{pid}.jsonl-
DIRECTORYIn which directory should the JSONL files be stored?str/opt/autopi/data-
FILE MODEIn which mode should new files be opened?stra-

Async File Writer#

FLUSH TIMEOUTTime limit for when the internal buffer should be written to disk.int1s
FLUSH THRESHOLDSize limit for when the internal buffer should be written to disk.int5242880b
BUFFER HIGH WATERMARKSize for buffer highwater mark
BUFFER SIZESize of the internal buffer.int52428800b

File Rotation#

SIZE THRESHOLDSize limit for when to rorate to a new file.int524288000b
INTERVALTime interval for when to rorate to a new
CRON EXPRESSIONCron expression for when to rotate to a new file.str0 0 * * *-

Jsonl Shipper#

ENABLEDEnable service for shipping JSONL files to cloud?boolFalse-


OLDER THANThe minimum age before fully harvested JSONL files are removed.int3600s
INTERVALHow often should cleanup of harvested JSONL files be performed?int3600s


PATHSWhere to crawl for JSONL files?str['/opt/autopi/data/*.jsonl']-


USERNAMEThe username for connecting to Kafka.strXXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX-
HOSTSKafka broker addresses from where to fetch the cluster metadata. The cluster metadata contain the actual Kafka brokers messages are published to.str['localhost:9093']-
PASSWORDThe password for connecting to Kafka.strXXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX-