How Matflow actually works

A methodology-level account of every stage, written against the real implementation — not a marketing simplification.

Overview

Experimental R&D is data-scarce: a typical campaign — catalysis, formulation, battery, polymer — produces tens, not thousands, of rows. Matflow's pipeline exists to make that scarcity workable: generate statistically faithful synthetic rows, prove the data synthesis is trustworthy before using it, train a model that can explain itself, and then search that model's response surface for candidates worth testing in the lab — ranking them by performance and cost. Every number the pipeline produces carries an evidence class (measured, computed, predicted, extracted, hypothesis, demo) and opens to an engine card naming exactly which engine produced it.

Data Synthesis

Matflow's Data Synthesis engine offers three modes:

• Matflow Neural — a conditional-GAN-style mode designed for tabular data with mixed continuous/categorical columns and multi-modal distributions. The strongest default for larger, more complex datasets.
• Matflow Hybrid — combines a GAN-style network with a copula transform, often more stable than Matflow Neural on smaller samples.
• Matflow Statistical — a fully statistical (non-neural) mode that fits marginal distributions and a correlation structure directly. Fastest, and the most defensible choice when rows number in the tens rather than hundreds.

Generated rows pass through post-hoc range clipping so no synthetic value falls outside the physically observed range of its column — a GAN can otherwise extrapolate a percentage column past 0–100%, for instance.

Quality evaluation

Synthetic data is never trusted by assumption. Every batch can be scored with:

• Quality Report — column-shape scoring (a Distribution Match Score for continuous columns, a Category Match Score for categorical) and column-pair correlation similarity.
• ML efficacy — an Indistinguishability Score measuring how easily a classifier can tell real from synthetic rows apart; the harder that is, the more faithful the synthesis.
• Dimensionality reduction — a Linear Map and a Structure Map projection of real vs. synthetic rows, so divergence is visible, not just scored.
• Privacy — Matflow's Privacy Shield checks record distance to the closest real row, an overfitting guard, and a duplicate check for exact near-duplicates, relevant the moment synthetic data is shared outside the originating lab.
• Outlier/anomaly detection — the Matflow Anomaly Detector, unsupervised and threshold-free, flagging the worst-fitting rows for manual review.

Predictive modeling

The predictor is the Matflow Predictive Ensemble — a five-model voting ensemble combining several tree-based regressors and a linear regressor — rather than a single model, because no one algorithm is reliably best across the small, heterogeneous datasets this platform sees. Every member is pinned to single-threaded execution internally; letting any one of them multithread while the ensemble itself is cross-validated causes severe oversubscription (a training run that should take under two minutes can otherwise take twenty).

Feature engineering ahead of training includes:

• A physics-derived transform layer (inverse, log, sqrt, square) applied per feature shape, plus a logit transform specifically for bounded-percentage targets.
• Latent components from the transform layer as additional features.
• Automated feature selection for dimensionality control.
• Optional Matflow composition descriptors (132 columns) for any input column holding a chemical formula.

Explainability is not an afterthought: Matflow Global Impact (summary, dependence, waterfall) and Matflow Local Impact both run against the fitted ensemble, plus permutation importance. Uncertainty is reported two ways — conformal prediction intervals (the 90th-percentile out-of-fold residual from the same cross-validation the metrics already compute, so it costs no extra training time) and a Matflow Anomaly Detector extrapolation flag on every prediction, so a number the model is guessing at looks different from one it has real support for.

Multi-objective optimization

Two search strategies are available: Matflow Adaptive Search, a surrogate-guided search, and Matflow Evolutionary Search, a genuine multi-objective genetic algorithm. Candidates are ranked by TOPSIS against the configured objective weights, and Pareto dominance is computed directly so the result is an actual trade-off front, not a single "best" answer. Compositional constraints (a set of columns that must sum to 100%, essential for formulations such as catalyst compositions) and per-feature bounds are enforced during candidate generation, not filtered out afterward.

TEA

The cost-aware optimizer, TEA, prices every candidate from raw material composition, equipment sizing, CapEx/OpEx factors and end-of-life & recycling economics — five separate costing modules feeding one net-value score. Equipment sizing uses the standard six-tenths power-law rule (cost ∝ (size / base_size)^0.6) when actual/base size are supplied, matching standard chemical-industry scale-up cost modeling.

Screening, design of experiments & active learning

Virtual screening ranks a candidate library by structural similarity (via the Matflow Cheminformatics Engine) to a reference molecule, with an optional SMARTS substructure filter. Design of Experiments generates full-factorial, Box–Behnken, central-composite, Latin Hypercube and Sobol designs — the inverse question to prediction: what should be measured next, not what a given formulation will do. Active learning closes the loop: the Matflow Active Learning Engine (a Gaussian Process-based Bayesian approach) proposes the next experiment batch from an existing optimization job's history, falling back to a heuristic surrogate when the full engine isn't available — the response always states which engine actually ran.

Limitations, stated plainly

• Small-data caveats. Every stage above is built to cope with tens of rows, but "cope with" is not "perform as well as with thousands." Confidence intervals widen and extrapolation flags fire more often exactly when data is scarcest.
• Synthetic data is not a substitute for real experiments. It preserves statistical structure well enough to train a model, but it cannot manufacture information that was never measured in the first place.
• Extrapolation risk. The ensemble is a strong interpolator within the training envelope and an honest, flagged guesser outside it — the extrapolation detector exists specifically because the model itself cannot tell you when it's guessing.
• Matflow Evolutionary Search and Matflow Adaptive Search find good fronts, not provably optimal ones. Both are heuristic searches; neither guarantees the global Pareto front has been found, only a good approximation of it.

References

The validation datasets and laboratory case study are documented in the platform's public benchmarks and evidence reports; the underlying methods used within the platform are drawn from the peer-reviewed work listed below. Links resolve through the DOI system.

• "High-throughput computational screening of cathode materials for Li-O2 battery." Computational Materials Science, 197, 110592 (2021). doi:10.1016/j.commatsci.2021.110592 ↗
• "Visualising multi-dimensional structure/property relationships with machine learning." Journal of Physics: Materials, 2(3), 034003 (2019). doi:10.1088/2515-7639/ab0faa ↗
• "Statistical Analysis and Discovery of Heterogeneous Catalysts Based on Machine Learning from Diverse Published Data." ChemCatChem, 11(18), 4537–4547 (2019). Source of the CADS oxidative-coupling-of-methane benchmark dataset. doi:10.1002/cctc.201900971 ↗
• "On the Quality of Synthetic Generated Tabular Data: Evaluation Metrics and Best Practices." Mathematics, 11(15), 3278 (2023). doi:10.3390/math11153278 ↗