Skip to main content

Machine Sections

Machine sections are optional acceleration structures at the bottom of a CSDB file. Readers must be correct without them.

Machine Zone​

When enabled, machine sections must appear after all human sections:

--- row_index:<table>:<primary_key_columns>
--- section_index                                                               
--- footer

Fixed Records​

Every machine section uses a fixed-width section title and fixed-width records.

Rules:

  • Section title lines are padded with spaces to exactly section_title_length bytes, including the --- prefix and section name.
  • Record keys are left-aligned and space-padded to machine_key_width.
  • Record values are right-aligned and space-padded to machine_value_width.
  • Every value, including text, is right-aligned.
  • All machine keys use machine_key_width and all machine values use machine_value_width from --- csdb.
  • The footer is read using footer_entry_count from --- csdb.
  • Machine sections do not contain section-local headers.

Parsing:

key   = line.slice(0, machine_key_width).trim()
value = line.slice(machine_key_width, machine_key_width + machine_value_width).trim()

Records use one key field and a fixed number of value fields defined by the section kind:

section_index record = key[machine_key_width] + value[machine_value_width] + LF
row_index strategy   = "strategy"[machine_key_width] + "sorted_binary"[machine_value_width] + LF
row_index record     = key[machine_key_width] + value[machine_value_width] + value[machine_value_width] + LF

Machine section title lines use:

section_title = raw_title.padEnd(section_title_length) + LF

Length-Based Lookup​

Machine sections store lengths instead of absolute offsets. Readers compute offsets in memory with prefix sums.

Lookup flow:

footer -> section_index count -> section lengths -> target section offset
row index -> row_number + row lengths -> target row offset

The footer is the final section. The --- csdb section declares the machine key width, machine value width, section title length, and footer entry count. Footer size is computed as:

section_title_length + footer_entry_count * (machine_key_width + machine_value_width + byte_length("\n"))

The footer stores only the number of records in section_index. Because section_index immediately precedes the footer, readers compute the section index length from that count and locate it by subtracting the computed section index length and footer length from the file size.

--- footer                                                                      
section_index_count                                                                                                    8

The section index byte length is computed as:

section_title_length
+ section_index_count * (machine_key_width + machine_value_width + byte_length("\n"))

Section Index​

The section index stores every section in file order. It is the only section directly located by the footer.

--- section_index                                                               
csdb                                                                                                                 320
table:workers:schema                                                                                                 900
table:workers:data                                                                                                   120
table:sessions:schema                                                                                               1400
table:sessions:data                                                                                                  220
row_index:workers:id                                                                                                 512
row_index:sessions:id                                                                                                512
footer                                                                                                               991

Section index records store:

<SECTION_NAME> <SECTION_LENGTH>

To find a section, readers sum section lengths before it. Readers may sum forward from the start of the file or backward from the end. The footer is included as the final section record.

Row Index Strategy​

CSDB v1 supports one row index strategy:

  • sorted_binary: fixed-width records sorted by key and searched with binary search.

Row index sections contain no local headers. The indexed columns and sort strategy are implied: v1 row indexes are primary-key indexes sorted by primary-key value.

Row index records store row numbers and row lengths, not byte offsets. Readers derive the row-index record count from the row-index section length and build row offsets in memory by summing row lengths in row-number order.