Tipsy
Compression
With ASCII Aux Files
Isolated Disc Outputs
- Speed here can be WILDLY different. I’m taking a tarball that is originally 51492 MiB.
| Algorithm | Default Size (MiB) | Max Compress Size (MiB) |
|---|---|---|
| gzip | 21057 | 20894 |
| zstd | 20273 | 18641 |
| lz4 | 30427 | 24737 |
| lzma | 18401 | 18385 |
| bz2 | 19345 | 19345 |
| xz | 18401 (SLOW) | 18385 |
| lrzip | 18433 | 18427 |
| lzop | 28625 | 24401 |
- lzma is the standout winner
- bz2 looks to be nearly as good, and is widely available
ChaNGa/Gasoline starlogs
Because star formation events happen once, and quantities associated with formation (such as the density of the gas particle the star formed from) do not change, it is inefficient to store these in the data structure for a star particle. It is much better to just write this out once, reducing the overall memory usage and the size of snapshots. ChaNGa and gasoline both use a common “starlog” format, typically written out as achOutName.starlog. Depending on the configuration options, this starlog can have the following format, with a couple possible sizes:
| Record Size | Gasoline starlog format | ChaNGa starlog format |
|---|---|---|
| 88 | 4-byte iorders, no H2 Cooling, no SFEventCrit | NONE |
| 96 | 4-byte iorders, H2 Cooling, no SFEventCrit | 8-bit iorders, no H2 cooling |
| 104 | 4-byte iorders, no H2 Cooling, SFEventCrit | 8-bit iorders, H2 cooling |
| 112 | 4-byte iorders, H2 Cooling, SFEventCrit | NONE |
ChaNGa starlogs
All Cases: 4 byte “record size” integer Then for every star formation event.
Cooling: 104 bytes
| Field | Type | Size |
|---|---|---|
| iOrdStar | int64_t | 8 |
| iOrdGas | int64_t | 8 |
| timeForm | double | 8 |
| rForm[3] | double[3] | 24 |
| vForm[3] | double[3] | 24 |
| massForm | double | 8 |
| rhoForm | double | 8 |
| TForm | double | 8 |
| H2FracForm | double | 8 |
Any other cooling mode: 96 bytes
| Field | Type | Size |
|---|---|---|
| iOrdStar | int64_t | 8 |
| iOrdGas | int64_t | 8 |
| timeForm | double | 8 |
| rForm[3] | double[3] | 24 |
| vForm[3] | double[3] | 24 |
| massForm | double | 8 |
| rhoForm | double | 8 |
| TForm | double | 8 |
Gasoline Starlogs
Gasoline can produce a slightly different output:
Cooling: 96 bytes
| Field | Type | Size |
|---|---|---|
| iOrdStar | int | 4 |
| iOrdGas | int | 4 |
| timeForm | double | 8 |
| rForm[3] | double[3] | 24 |
| vForm[3] | double[3] | 24 |
| massForm | double | 8 |
| rhoForm | double | 8 |
| TForm | double | 8 |
| H2FracForm | double | 8 |
Any other cooling mode: 88 bytes
| Field | Type | Size |
|---|---|---|
| iOrdStar | int | 4 |
| iOrdGas | int | 4 |
| timeForm | double | 8 |
| rForm[3] | double[3] | 24 |
| vForm[3] | double[3] | 24 |
| massForm | double | 8 |
| rhoForm | double | 8 |
| TForm | double | 8 |
If SFEVENTCRIT is enabled, this adds another two doubles after TForm: tcool and tdyn