1.2.3 — Data Types and Feature Representation

1. Meaning comes before storage format

Imagine a CSV column containing 101, 205, and 307. Technically, these are integers. But what do they mean: water consumption, room numbers, or equipment-type codes? For consumption, a difference or average may be meaningful. The average room number, 204.3, describes nothing useful. A column's file type does not determine its modelling meaning.

Separate three levels. The physical type describes storage (string, integer, float, timestamp). The semantic type describes the domain meaning (measurement, count, category, date, identifier). The task role describes how the field is used (input feature X, target y, grouping key, or administrative field). The same field can play different roles in different tasks.

Ask: which operations make sense for this value? Can it be compared, added, averaged, or ranked? The answer often reveals its meaning better than the column name.

2. Measurement scales: what do numbers mean?

A classic classification of measurement scales helps prevent unsupported assumptions. A nominal scale distinguishes classes only. An ordinal scale adds a meaningful ranking. An interval scale supports differences but has an arbitrary zero. A ratio scale has a natural zero, so ratios are meaningful.

Temperature in Celsius is an interval scale: the difference between 10 and 20 degrees equals the difference between 20 and 30, but 20 °C is not twice as warm as 10 °C. Mass in kilograms is a ratio scale: 20 kg is twice 10 kg. A rating such as low / medium / high is ordinal: the order is known, but the gaps need not be equal.

3. Numeric features: measurements and counts

Continuous features describe measurements that can take values across a range: temperature, pressure, area, operating time. Discrete features usually count events or objects: requests, floors, defects. The distinction affects range checks, visualisation, transformations, and sometimes model choice.

Check units, precision, and physically plausible bounds. Mixing °C with °F or square metres with square feet creates plausible-looking but incorrect data. Investigate outliers separately: 9,000 may be a typo, a rare real event, or a different unit. Do not delete it automatically; establish where it came from.

A missing value also carries meaning. An empty cell, the string “not available”, zero, and a sentinel such as −999 are not equivalent. Sometimes the absence of a measurement is informative: a sensor may disconnect more often under a particular operating condition. A missingness indicator can help, but replacing unknown values with an arbitrary number needs justification.

4. Categories: nominal and ordinal

Nominal categories have no natural order: wall material, equipment type, city, or contact channel. Ordinal categories have a meaningful sequence: low, medium, high; beginner, advanced, expert. Encoding these levels as 1, 2, and 3 does not make the distances between them measured quantities.

A common mistake is to encode building types as 1, 2, and 3 and pass them to an algorithm that interprets distances. The model may then treat type 3 as “greater” than type 1 and assume that the gap from 1 to 2 equals the gap from 2 to 3. This geometry is artificial for nominal data.

5. Categorical encoding: options and trade-offs

One-hot encoding creates a separate binary column for each category. Three building types become three indicators. This avoids imposing an order but increases dimensionality. With thousands of unique values, the matrix becomes wide and sparse; consider category frequencies, grouping rare values, hashing, or algorithms with native categorical support.

Ordinal encoding assigns integer codes. It is appropriate for a genuinely ordered scale when the order is explicit. For nominal categories, suitability depends on the algorithm: trees and models with native categorical support may treat categories differently from a linear model or distance-based method. Do not choose an encoding only because it is compact.

Frequency encoding replaces a category with its frequency. Target encoding uses target statistics within each category. These methods can help with high cardinality but are especially vulnerable to leakage and overfitting. Calculate statistics on training data only; target encoding commonly needs out-of-fold estimation and smoothing. Define how unseen categories will be handled at inference time.

6. Dates, time, and cyclic patterns

A timestamp may include a date, time, time zone, and measurement precision. Before extracting features, identify which event the timestamp represents: request creation, actual work start, system registration, or record ingestion. These are different moments. Use a consistent time standard and account explicitly for the local calendar.

A date can yield year, month, weekday, hour, holiday, season, or elapsed time since an event. Use only information available at prediction time. For periodic quantities, a raw hour number creates an artificial gap between 23 and 0. Cyclic encoding maps values onto a circle, making neighbouring hours close in both sine and cosine coordinates.

7. Text and identifiers

A short label (“fault”, “scheduled inspection”) is a categorical feature. A free-form service description is text. Text representations include bag-of-words, TF-IDF, n-grams, and embeddings. Simple representations work well for some classification tasks; embeddings can capture semantic similarity but require additional computation and quality controls.

An identifier is usually a key, not a description of an object. A customer, device, or ticket ID may let a model memorise entities instead of learning a transferable pattern. Yet IDs can be useful for group splitting, joining tables, auditing, and monitoring. The right decision is not “always drop it”; separate its administrative role from its role as an input feature.

8. Scaling and transformations

Many algorithms are sensitive to scale. If area is measured in thousands and temperature in tens, distances and gradients may be dominated by area. StandardScaler centres values and divides by standard deviation; MinMaxScaler maps them to a chosen range; RobustScaler uses the median and interquartile range and is less affected by extreme values.

Decision trees and tree ensembles usually do not require scaling because splits use feature thresholds. A logarithm may reduce skew in positive values, but zeros and negative values need attention. A transformation should match the feature's nature and the task rather than being a ritual preprocessing step.

9. Missingness, outliers, and leakage: three sources of false confidence

Missingness may be random, depend on other observed fields, or relate to the unknown value itself. Study its mechanism and distribution across groups and time. Mean, median, most-frequent category, an explicit “unknown” category, and a missingness indicator are different hypotheses, not interchangeable buttons.

Leakage occurs when preprocessing or feature construction uses information unavailable in the real prediction scenario. For example, predicting tomorrow's energy use while including tomorrow's actual occupancy is invalid. Another example is computing means and category vocabularies on the full dataset before splitting train and test. The metric becomes optimistic, but the model cannot reproduce it in deployment.

10. Feature specification and a reproducible workflow

For every field, record its meaning, physical and semantic type, units or allowed values, source, availability time, missingness rate, transformation, unseen-value behaviour, and leakage risk. This makes data preparation auditable: another practitioner can understand not only what was done, but why.

Any parameter learned from data — an imputation mean, scale, category vocabulary, frequency, or target statistic — must be fitted on train only. Apply the same fitted transformer to validation, test, and new data without fitting again. In scikit-learn, Pipeline and ColumnTransformer help enforce this discipline.

Rule of thumb: split data according to the deployment scenario, fit transformations on train, then apply them to the other partitions. Never let future or test data influence preprocessing of the past.