Skip to main content

Cloud Storage

NameDescriptionTypeDefaultUnit
DEFAULT RETURNERWhich cloud returner should be used as default? Only change this value if you know what you are doing.strcloud-

Https#

Cache#

NameDescriptionTypeDefaultUnit
MAX BATCH SIZEThe maximum amount of entries to include in an upload batch.int1000-
MAX RETRYThe maximum retry attempts before transferring batch to fail queue. Retry and fail queues are disabled if set to zero, meaning that pending will block until success.int30-
FAIL TTLHow many days of retention before a fail queue is deleted?int30d

Endpoint#

NameDescriptionTypeDefaultUnit
URLThe endpoint URL of the API where data is pushed to.urlhttps://api.autopi.io/logbook/storage-
AUTH TOKENThe authentication token used in HTTP header when pushing data to the API.strXXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX-

Upload#

NameDescriptionTypeDefaultUnit
INTERVALHow often should cached data be uploaded to the cloud endpoint?float60s
SPLAYThe maximum amount of time in seconds between consecutive uploads of batches. The actual delay is a randomly generated value between zero and the splay value. In error scenarios, the delay is increased by an incrementing factor.int10s
TIMEOUTHow long to wait for server to respond?int60s
EVENT REGEXIn addition to repeated interval, upload data to cloud when event(s) occur.str^system/power/(sleep|hibernate|reboot)-

Jsonl Returner#

NameDescriptionTypeDefaultUnit
FILENAME FORMATHow should the JSONL files be named?str{now:%y%m%d%H%M%S}_{pid}.jsonl-
DIRECTORYIn which directory should the JSONL files be stored?str/opt/autopi/data-
FILE MODEIn which mode should new files be opened?stra-

Async File Writer#

NameDescriptionTypeDefaultUnit
FLUSH THRESHOLDSize limit for when the internal buffer should be written to disk.int5242880b
FLUSH TIMEOUTTime limit for when the internal buffer should be written to disk.int1s
BUFFER SIZESize of the internal buffer.int52428800b
BUFFER HIGH WATERMARKSize for buffer highwater mark warning.int-b

File Rotation#

NameDescriptionTypeDefaultUnit
SIZE THRESHOLDSize limit for when to rorate to a new file.int524288000b
CRON EXPRESSIONCron expression for when to rotate to a new file.str0 0 * * *-
INTERVALTime interval for when to rorate to a new file.int-s

Jsonl Shipper#

NameDescriptionTypeDefaultUnit
ENABLEDEnable service for shipping JSONL files to cloud?boolFalse-

Cleanup#

NameDescriptionTypeDefaultUnit
INTERVALHow often should cleanup of harvested JSONL files be performed?int3600s
OLDER THANThe minimum age before fully harvested JSONL files are removed.int3600s

Input#

NameDescriptionTypeDefaultUnit
PATHSWhere to crawl for JSONL files?str['/opt/autopi/data/*.jsonl']-

Output#

NameDescriptionTypeDefaultUnit
HOSTSKafka broker addresses from where to fetch the cluster metadata. The cluster metadata contain the actual Kafka brokers messages are published to.str['localhost:9093']-
USERNAMEThe username for connecting to Kafka.strXXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX-
PASSWORDThe password for connecting to Kafka.strXXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX-