Data Logger / Field guide
Data logging turns observations into a sequence that someone can interpret. Make time, units, source identity, and quality explicit so a later analysis can distinguish a real change from a recording problem.
Start with the purpose
A measurement needs more than a value. A reader should be able to identify the measured quantity, its unit, when it was observed, the source that produced it, and any known limitation. If these meanings live only in the person who configured the logger, they can disappear when the data moves to another team.
Choose timestamp representation and clock quality separately. An unambiguous format makes a time value easier to interpret, but it does not establish that the source clock was correct. Consider recording arrival time or a sequence number where delayed delivery and missing samples are operationally important.
Treat units as part of the schema. If an export changes a unit or applies a conversion, preserve enough provenance to explain what happened. A derived measurement should not quietly replace its source value without an agreed rule. The same discipline applies to calibration references and quality flags: record what they mean and who maintains them.
Design for missing and questionable values. A gap, an out-of-range observation, and a legitimate zero represent different situations. Choose explicit handling for each and test those paths with a small sample set. Review how the exported data looks to an analyst who did not build the collection system.
Capacity planning belongs in the design as well. Estimate the production rate, record size, retained duration, and number of copies. Then measure representative data and revise the estimate. Compression, indexes, transfer, and retrieval can affect the plan, so keep assumptions visible instead of treating a raw byte calculation as a complete operating budget.
Four decisions to make
Separate observation time from arrival time where delays matter. Keep the chosen time basis documented.
Store the quantity and unit together, and make any conversion or scaling step reviewable.
Define missing, invalid, suspect, and accepted states so downstream code can handle them deliberately.
Estimate normal production, bursts, retained copies, and recovery space using measured examples.
Build measurement records with clear timestamp roles, explicit units, quality states, and identities that survive retries and restarts.
Read Data logger design: timestamps, units, and data quality
Estimate logging volume from event rate, record size, retention, and measured overhead, then account for the work around storage.
Read Logging storage costs: build a transparent capacity estimateStart with the fundamentals, then follow the signal that matters to your work.