Loading page...
Getting the next page ready.
Loading page...
Getting the next page ready.
ink on the page —
Deep dives into AI systems, Data Science war stories, and IoT engineering from the field — no fluff, just signal.
from the notebook
Most RAG tutorials stop at LangChain + Pinecone. Real production systems demand proper chunking strategies, hybrid retrieval (dense + sparse), reranking layers, and hallucination guardrails. Here's what actually matters at scale.
LSTM models achieve suspiciously low MSE on test sets — until you realize data leakage, look-ahead bias, and improper walk-forward validation are to blame. Most tutorials scale the entire dataset before splitting, feeding future min/max stats directly into past training sequences. A rigorous breakdown of common time-series ML pitfalls, walk-forward cross-validation, and why persistence baselines are hard to beat.
⚠️ Core Pitfalls Uncovered:
Variational Autoencoders learn the 'normal' distribution of industrial sensor data, flagging reconstructions with high ELBO loss as anomalies. A practical walkthrough using PyTorch, FastAPI, and MQTT for real-time edge inference.
When your positive class is 1.3% of records, accuracy is a useless metric and SMOTE alone won't save you. Explore SHAP-driven feature selection, threshold calibration, and cost-sensitive XGBoost for real-world class imbalance.
From ReAct to Chain-of-Thought, structured output enforcement, and tool-calling workflows — this deep dive covers the prompt engineering primitives you need to build reliable, controllable AI agents in 2025.
Cloud inference adds latency and bandwidth costs to IoT systems. Walk through model quantization, ONNX export, TFLite conversion, and deployment to Raspberry Pi and ESP32 with sub-50ms inference benchmarks.
questions & answers
My primary stack is Python with PyTorch or TensorFlow/Keras for deep learning, Scikit-learn for classical ML, FastAPI for serving models as REST APIs, and Streamlit for rapid prototyping UIs. For RAG systems I use Sentence-Transformers + FAISS or Pinecone, and Docker for containerised deployment.
Hallucination mitigation is multi-layered: (1) proper document chunking with semantic overlap so context windows stay coherent, (2) a reranker (cross-encoder) to ensure only truly relevant passages reach the LLM, (3) confidence thresholding on retrieval similarity scores, and (4) structured output enforcement with Pydantic so the LLM can't fabricate schema fields.
IoT introduces real constraints that typical ML pipelines ignore: severely limited compute at the edge, lossy/asynchronous data streams, concept drift from sensor degradation, and strict latency requirements. Models must be quantised, pruned, and tested against ONNX/TFLite runtimes. Data pipelines also need dead-reckoning for intermittent connectivity.
Accuracy is almost always misleading on imbalanced data. I rely on F1 score (macro/weighted), ROC-AUC, Precision-Recall curves, and calibration plots. For regression I use MAE alongside RMSE to understand outlier sensitivity. SHAP values give feature-level explanability, and I always do a confusion matrix error analysis to understand *where* the model fails.
Yes — my GenAI writing focuses on practical engineering rather than hype. Topics include LLM agent design patterns, prompt engineering for structured outputs, context window management, embedding model selection, and building production RAG systems with proper evaluation metrics like RAGAS.
Most articles target intermediate practitioners who know Python and have some ML fundamentals. I assume familiarity with NumPy/Pandas and basic model training. Where I introduce advanced topics (VAEs, SHAP, ONNX quantisation), I link to prerequisite resources so the content remains accessible with the right background.