Feature engineering is one of those things that every ML practitioner knows matters enormously, but few people have a principled system for. You get a new dataset, you look at the distributions, you think “this column is skewed – I should probably log-transform it,” and then you do it. Or you forget. Or you apply it where it shouldn’t apply. Or a new team member inherits the notebook and has no idea why half the transforms are there.
I built FeatureIQ because I kept running into the same problem: the decisions I was making during feature engineering were implicitly correct – backed by real ML literature – but they were locked in my head. They weren’t reusable, they weren’t explainable to others, and they certainly weren’t automatic.
FeatureIQ is my attempt to fix that. It’s an open-source Python library that automates feature engineering by encoding expert knowledge into a structured, citation-backed rule system – and then using dataset profiling to apply the right transformations automatically, based on your actual data, problem type, and algorithm.
The Core Problem: Three Things Interact at Once
Here’s the thing about feature engineering decisions: they’re never just about the data in isolation.
Whether to log-transform a skewed column depends on:
- The data – is it actually skewed? Does it have zeros or negatives?
- The algorithm – log transforms matter for linear models and distance-based methods, but are largely irrelevant for tree-based models
- The problem type – the semantics of your target variable change what’s appropriate
FeatureIQ captures this interaction systematically. Every transformation decision is a rule in a structured ontology, and every rule is backed by a citation from ML literature.
How It Works
Step 1: Dataset Profiling
When you call fit(), FeatureIQ profiles your dataset automatically – computing meta-features like skewness, cardinality, missing rate, variance, and correlation structure for every column.
Step 2: Rule-Based Recommendation
The profiler output feeds into a rule engine that consults the ontology: a collection of YAML-defined rules, each encoding a transformation recommendation with conditions and a literature citation.
For example: Apply log1p transformation to continuous features with skewness > 1.0, when the algorithm is distance-based or linear, and the feature has no negative values.” – backed by Géron (2019), Hands-On Machine Learning.
A meta-learner (trained on OpenML datasets) also contributes, acting as a learned fallback for edge cases where rules don’t give a clear answer.
Step 3: Pipeline Construction
Recommended transforms are assembled into a scikit-learn–compatible pipeline and applied via transform(). The pipeline handles column exclusions, overrides, and pre/post injection points – so you stay in control.
Key Capabilities
1. sklearn-Compatible Out of the Box
FeatureIQ implements fit, transform, and fit_transform – which means you can drop it directly into a sklearn.pipeline.Pipeline:
from sklearn.pipeline import Pipeline
from sklearn.linear_model import LogisticRegression
from featureiq import FeatureIQ
pipe = Pipeline([
(“features”, FeatureIQ(algorithm=”logistic_regression”, problem_type=”binary_classification”)),
(“model”, LogisticRegression())
])
pipe.fit(X_train, y_train)
pipe.predict(X_test)
No special wrappers. No custom interfaces. It just works where sklearn works.
2. Composability
Real datasets are never clean slates. FeatureIQ gives you fine-grained control:
fiq = FeatureIQ(
algorithm=”xgboost”,
problem_type=”binary_classification”,
exclude_columns=[“id”, “user_token”],
override_transforms={“revenue”: [“log1p”]},
pre_transforms=[my_custom_cleaner],
post_transforms=[my_business_logic_step]
)
3. Cross-Column Awareness
FeatureIQ doesn’t look at each column in isolation. For linear models, it detects multicollinearity using Variance Inflation Factor (VIF) and handles correlated features appropriately. For tree-based models, it skips this step entirely – because tree-based models handle multicollinearity natively, and penalizing correlated features there would be wrong.
That kind of context-sensitivity is exactly the kind of thing that gets missed in hand-rolled pipelines.
What It Covers
FeatureIQ currently supports:
14 algorithms across five families:
- Tree-based: XGBoost, LightGBM, Random Forest, AdaBoost, Bagging
- Linear: Logistic Regression, Linear Regression, Ridge, Lasso
- Distance-based: SVM, KNN
- Neural: MLP
- Ensemble: AdaBoost, Bagging
5 problem types: binary classification, multiclass classification, regression, time-series forecasting, anomaly detection
The algorithm and problem type together determine which rules are active for a given run – which means the same dataset processed with algorithm=”xgboost” and algorithm=”knn” will produce meaningfully different transformation pipelines.
Why This Approach
I want to be honest about what FeatureIQ is and isn’t. It isn’t magic. It doesn’t discover new transformations. It doesn’t replace understanding your data.
What it does is encode the decisions that experienced practitioners already know – and make those decisions repeatable, auditable, and context-aware. The ontology is explicit. The citations are real. The ablation analysis tells you whether the transforms actually helped in your specific situation.
I think there’s something valuable in that honesty. ML pipelines accumulate technical debt quickly, and a lot of that debt comes from transformations that were added without documentation and can’t be safely removed later. FeatureIQ tries to reverse that trend by making the “why” a first-class citizen.
What’s Next
The current version covers the core workflow well, but there’s a lot of room to grow. I’m actively working on:
- Expanding the ontology with more algorithm-specific rules
- Better time-series support with additional temporal feature transforms
- A more expressive meta-learner trained on a wider OpenML sample
- Interactive explain_report() output for notebook environments
If you work in ML and want to contribute – whether that’s adding a rule, improving the profiler, or just writing a test – contributions are very welcome. The project is on GitHub and the structure is designed to make extension straightforward.
References
[1] A. Géron, Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow (2019), O’Reilly Media
[2] M. Kuhn and K. Johnson, Applied Predictive Modeling (2013), Springer
[3] H. Guo and B. Yuan, A survey of feature selection and feature extraction methods for machine learning (2014), IEEE Symposium on Computational Intelligence and Data Mining
[4] J. Vanschoren et al., OpenML: Networked Science in Machine Learning (2014), ACM SIGKDD Explorations Newsletter
FeatureIQ is open source and available on PyPI and GitHub. If you try it out – or have thoughts on the ontology design – I’d genuinely love to hear from you.