What Is Data in Machine Learning?

1. Data are records about the world, not the world itself

In machine learning, data are recorded information about entities, events, or processes: sensor readings, photographs, support messages, or transaction records. Data help us study reality, but they are not reality itself.

Consider a building. The building is the entity. Room temperatures, electricity-meter readings, operating schedules, and equipment photographs are different digital records about it. Each describes only one aspect.

A record is produced by a procedure: someone chooses what to measure, selects an instrument, sets a sampling frequency, stores the result, and may transform it. Ask not only “what number is stored?” but also “what does it mean, and how was it produced?”

The value 125 alone tells us little. It could mean 125 kWh per day, 125 kW at one moment, a temperature, or a device ID. We need the metric, unit, time, and collection method.

2. Entity, unit of observation, observation, and dataset

Before analysis, decide what one record represents. Distinguish four terms:

• Study entity: what is being studied, such as a building, engine, or customer.
• Unit of observation: the element represented by one record, such as a building-day, sensor-hour, or service request.
• Observation: the actual recorded entry for that unit.
• Dataset: an organized collection of observations assembled under consistent rules.

One entity can produce many observations. An hourly sensor produces 24 records per day for one building. These are not 24 buildings; they are 24 time-based observations of one entity.

If hourly records from one building are randomly split between training and test sets, similar conditions may appear in both. Evaluation can become overly optimistic. Depending on the task, split by building, time period, or another independent group. We will return to this in lesson 1.2.4.

3. Forms of data

Data are not always tables. Three common forms are useful for an introduction.

Structured data

These follow a predefined schema: rows, columns, field names, and value types. An energy table might contain building ID, date, consumption, and outdoor temperature.

Semi-structured data

These have recurring organization but a variable set of fields. JSON sensor events are an example: one message contains temperature and humidity, another temperature and an error code.

Unstructured data

These lack a simple universal table schema: free text, photographs, audio, and video. Examples include equipment-defect photographs and engineers’ notes.

A project may combine all forms. A table stores equipment IDs, JSON stores telemetry, and photographs document faults. Linking them requires reliable identifiers and timestamps. A faulty match can associate one device’s image with another device’s readings.

4. Sources, measurement, and time context

Sources include sensors, information systems, experiments, surveys, expert observations, public repositories, and human annotation. A source affects which entities enter a dataset and which cases remain unseen.

In energy monitoring, kWh measures energy, while kW measures power. They are not interchangeable. Determine whether a value is a cumulative meter reading or energy consumed during an interval, such as one hour.

A record marked “08:00” needs a date and time zone. Does it mark the start or end of an interval, or the moment a message arrived? During daylight-saving transitions, a local hour may repeat or be skipped.

Data may be processed: units converted, duplicates removed, errors filtered, tables joined, missing values imputed, or labels added. Document these operations. Do not silently replace a missing value with zero: zero may be a real measurement, while missing means unknown or not captured.

5. Quality, reliability, and limitations

Quality cannot be judged only by row count. Ask: does this dataset reliably support this particular task? A dataset may suit one purpose and be inadequate for another.

• Completeness: are important fields, days, seasons, or entity types missing?
• Accuracy: are sensors working and units and conversions correct?
• Consistency: are fields interpreted the same way across systems?
• Uniqueness: were events loaded more than once?
• Timeliness: do data reflect current processes and equipment?
• Annotation quality: are label definitions clear and errors uncommon?

If weekend records are absent, a model may learn only weekday patterns. If a sensor reports Fahrenheit while the system assumes Celsius, values are present but wrong. A duplicate event may receive extra weight.

Do not automatically delete every missing value, outlier, or duplicate. Unusually high consumption may be a faulty meter—or a real emergency. Investigate the cause, assess its impact, and document the decision.

6. Representativeness, sampling bias, and changing data

The target population is the full set of entities or situations to which conclusions or a model should apply. A sample is the collected subset. A sample is representative not because it is large, but because it reflects task-relevant characteristics of the target population.

If a model is intended for all buildings but the dataset contains only modern offices in one city, schools, hospitals, and older homes may be missing. This is sampling bias: selection systematically excludes or under-represents part of the target population.

Now consider a different case: the original data represented buildings well, but new heating systems, tariffs, and operating schedules were introduced years later. Conditions changed. This is distribution shift—a difference between development/evaluation data and data encountered in later use.

These phenomena are related but not identical. The first often arises during data collection design; the second can emerge after deployment. A more complex algorithm does not automatically fix poor coverage or changed real-world processes.

Compare data with the intended deployment setting: which entity types, periods, and seasons are included? Which groups are missing? How do processes change? Do not judge suitability by row count alone.

7. A dataset datasheet

A dataset datasheet (or dataset card) describes where data came from, what they contain, how they were processed, and which uses are appropriate or inappropriate. It helps prevent data from being used outside their context.

A practical minimum datasheet includes:

This is an educational template, not universal legal advice. If data contain personal information or can identify people, separately check lawful collection grounds, consent where required, access restrictions, and retention. Removing names alone does not guarantee anonymity.

Good documentation answers four questions: what does a row represent; how was each value obtained; which population is covered; and which conclusions are not supported?

8. Practice: describe a dataset and find weak points

You receive a file with daily energy consumption for 40 buildings over one year. It contains a building code, date, and consumption value, with no further explanation.

1. State a possible task: analyze seasonality or forecast next-day consumption.
2. Define the unit of observation. One building-day is a likely hypothesis, but confirm it.
3. Complete the datasheet. If information is unknown, write “unknown”; do not invent an answer.
4. Ask the owner at least three questions: which units are used; what does the date mean and which time zone applies; which buildings or days are missing and why?
5. List limitations: one year may miss rare weather; 40 buildings may be too similar; unknown processing makes values hard to verify.

Do not fill unknowns by guessing. Separate confirmed facts from assumptions and open questions.