A data logger is only as useful as the meaning of its records. A long list of numbers can look precise while leaving basic questions unanswered: when was a value observed, what does it measure, which unit applies, and was the source functioning normally? These decisions belong in the record design before the first dashboard is built.

This guide develops a practical measurement record and follows it through delayed delivery, missing values, unit changes, and validation. The examples are illustrative engineering choices, not product specifications. For a broader introduction to capture and review workflows, begin with the data logger overview.

Describe the observation before choosing the format

Start with a plain-language statement such as “the room sensor reports a temperature reading when a scheduled observation succeeds.” Clarify the source, the measured quantity, and the event that creates the record. Is the value an instantaneous reading, an average over an interval, or a total since the previous reset? Similar numbers can have very different meanings.

Write a data dictionary alongside the record format. For each field, define its type, unit, allowed absence, and interpretation. Include the owner who can answer questions about the source. A dictionary makes future changes discussable: reviewers can see whether a proposed modification changes the physical meaning, only the representation, or both.

A compact example record

{
  "schema_version": "1",
  "source_id": "room-sensor-c",
  "session_id": "boot-c72",
  "sequence": 418,
  "observed_at": "2024-01-04T09:14:22.340Z",
  "received_at": "2024-01-04T09:14:25.110Z",
  "quantity": "temperature",
  "value": 21.4,
  "unit": "Cel",
  "quality": "observed"
}

The field names are an example contract for this article. Document what each field means in your implementation. In particular, decide whether a session identifies a device boot, a collection job, or a continuous measurement interval; do not switch between those meanings without changing the contract.

Give every timestamp a job

Distinguish observation time from receipt time. The first describes when the source says the observation occurred. The second describes when the receiving component accepted the record. In the example, their difference is 2.770 seconds, but that arithmetic alone does not prove network latency: the clocks and the observation process also affect the comparison.

RFC 3339, Date and Time on the Internet defines a timestamp format with a date, time, and UTC relationship. A value ending in Z represents UTC; a numeric offset can also express that relationship. The format provides an interoperable representation, not evidence that a device’s clock is accurate.

Choose a consistent representation for stored observations and show local time when it helps a person review them. Preserve the source timestamp when normalizing its representation. If the source cannot establish a trustworthy observation time, make that state explicit rather than copying receipt time into the observation field and pretending the two are equivalent.

Record clock uncertainty honestly

Define how the system represents an unsynchronized clock, a clock reset, or an observation timestamp that fails validation. An explicit clock_status field can be useful if its values have a clear meaning. Avoid producing more decimal places than the source supports and then letting presentation imply corresponding accuracy.

For duration measurements within one process, choose a clock intended for elapsed intervals and document it. For comparisons across devices, document the synchronization assumptions. Keep these concerns separate so a convenient timestamp string does not quietly become a universal timing guarantee.

Make units part of the data contract

A field named value is incomplete without a quantity and unit convention. Temperature, energy, distance, and rate measurements cannot be safely interpreted through column position or a dashboard label alone. Include the unit in each record or bind the field to an explicit versioned schema that travels with the data.

In the example contract, Cel is the documented code for degrees Celsius. Choose your own code convention deliberately and use it consistently. If a downstream system expects a different representation, convert through a named transformation and retain enough provenance to explain the result.

Do not silently change a field from one unit to another while leaving its schema version unchanged. A unit migration should have an effective point, a conversion rule, and a way to distinguish old records from new ones. Test exports as well as dashboards, because a spreadsheet or scheduled report may bypass the display logic that applies a conversion.

Keep missing, invalid, and estimated values distinct

Zero is a valid measurement in many contexts. Using it to mean “no reading” can hide a disconnected source or create a false event. Define an explicit missing representation and a reason code that explains what the collection process observed.

A practical contract might distinguish observed, missing, invalid, and estimated. These labels need definitions rather than intuition. An out-of-range value could be a genuine unusual condition or a bad reading; the record should preserve the observation and the validation result long enough for the intended review.

If analysis fills a gap, store the derived value in a separate output with its method identified. Do not overwrite the original absence. A reviewer comparing raw collection with a report should be able to see which values came from the source and which came from later processing.

Expect retries, delays, and restarts

Design a record identity before adding retry behavior. A combination of source, session, and sequence can be one workable approach when its uniqueness assumptions are documented. Another system may use a separately assigned event identifier. The important choice is whether the receiver can recognize the same observation arriving again.

A sequence number can help expose gaps within a session, but it needs a reset rule. If a device restarts and begins counting again, the session boundary should distinguish new observations from old ones. Do not interpret a lower sequence number automatically as a transport error without checking that boundary.

Keep late arrival separate from duplicate arrival. A delayed observation can be new information even when its timestamp is older than records already displayed. Decide whether reports accept late data, revise prior results, or close an interval after a documented cutoff. The Log Mic workflow guide connects these record-level decisions to the wider process of investigation and review.

Walk through a mixed-unit failure

Imagine a facilities team that receives temperature readings from two device models. One exports Celsius and the other exports Fahrenheit. An early collector stores only a number and a device label. A chart then plots both sources against one axis, making one room appear dramatically warmer than the other.

The first fix is to identify and preserve the source unit. The team updates the contract, stores normalized values in a clearly identified derived field, and checks a few known conversions. It also reviews historical exports. Where the original unit cannot be established, the records remain marked as uncertain instead of being assigned a convenient assumption.

Next, a device reconnects after an outage and sends buffered observations. The team confirms that the chart uses observation time for the measurement timeline and receipt time for delivery diagnostics. A replay check verifies that retries do not create duplicate points. These changes address two different faults: ambiguous measurement meaning and ambiguous event handling.

Validate the contract at the boundaries

Run checks when records enter the system and when they leave it. Verify required fields, timestamp parsing, unit codes, type consistency, and the allowed quality states. Include examples of missing measurements, a restart, a late record, and a retry. These cases exercise the contract more usefully than many copies of the same normal reading.

Keep invalid records in a controlled review path only when that retention serves a defined purpose. Record an error category and the minimum context needed to diagnose the issue. Avoid copying unrelated identifiers or full payloads into general logs. The redaction and retention guide describes how to connect that diagnostic value with concrete handling rules.

When a field changes, check a representative downstream report as well as the collector. A parser accepting a record does not establish that a person will interpret it correctly. Review the labels, sorting, rounding, and missing-value display that shape the final decision.

Conclusion: preserve meaning through the whole journey

Useful data logging depends on explicit quantities, units, timestamp roles, quality states, and record identities. Document those choices, retain uncertainty, and test ordinary failure cases at system boundaries. A reviewer should be able to explain each value and its limitations without reverse-engineering the collector that produced it.