# Hikmah Technologies

> Production AI on your own data — lean data platforms and approval-gated AI workflows, internals public

Arshad Ansari is a solo data and AI engineer based in Navi Mumbai, India, working with
B2B SaaS and product companies worldwide (30–200 people, zero to three data people, no
senior platform owner). The practice sells one thing: production AI on your own data —
the data-engineering rigour that makes an AI workflow trustworthy, built by the person
who will also operate it. Engagements run $3,000 for a fixed-price audit to $15–40k for a
full build, against the $150–200k senior hire that is the usual alternative.

What makes the claims checkable: the internals are public. 3 systems I built and run are documented
with real numbers rather than case-study prose — 4,912 pipeline partitions at
99.9% success, and, in the archived open-source AEGIS v1, 49 workflow types over
72 permission-gated tools, where every risky action stops for approval before it runs.

## Start here

- [Home](https://hikmahtechnologies.com/): The position, the proof numbers, and who should not hire me
- [Data Engineering Consultant](https://hikmahtechnologies.com/fractional-data-engineer/): The data half: what an independent, fractional data engineering consultant does, costs, and when to hire full-time instead
- [AI Automation Consultant](https://hikmahtechnologies.com/ai-automation-engineer/): The AI half: shipping LLM workflows that survive production, and the alternatives to hiring me
- [DuckDB consulting](https://hikmahtechnologies.com/duckdb-consulting/): A fixed-price, one-week assessment of whether a workload fits DuckDB, with the full production-fit checklist published on the page
- [How I work](https://hikmahtechnologies.com/how-i-work/): Scoping, pricing, communication and what I expect from a client
- [FAQ](https://hikmahtechnologies.com/faq/): Direct answers on rates, contracts, timelines and working across time zones

## Free and ungated — no email required to read

- [Data Platform Teardown](https://hikmahtechnologies.com/data-platform-teardown/): The 18 checks from the $3,000 audit, published in full
- [AI Workflow Teardown](https://hikmahtechnologies.com/ai-workflow-teardown/): 17 questions to ask before an LLM workflow is trusted in production: control, reliability, cost, trust
- [Warehouse cost calculator](https://hikmahtechnologies.com/snowflake-cost-calculator/): Estimate your warehouse spend against a lean rebuild before committing to an architecture
- [Your Snowflake warehouse may never be sleeping](https://hikmahtechnologies.com/idle-warehouse-calculator/): The finding, the arithmetic and the SQL: a warehouse queried faster than its auto-suspend window never suspends, is billed as always-on, and looks correctly configured
- [Send me your cost export](https://hikmahtechnologies.com/send-your-cost-export/): Send a Snowflake cost export and get a written reply naming what to investigate first — free, no call
- [warehouse-teardown](https://github.com/hikmahtech/warehouse-teardown): Open-source read-only CLI that answers five of the 18 audit questions from Snowflake's own metadata and prints the rest as a checklist
- [Local-First Analytics (book)](https://hikmahtechnologies.com/book/): The book on running analytics without a cloud warehouse; chapter one is free to read on-site

## Proof — verifiable claims, not testimonials

- [Systems](https://hikmahtechnologies.com/systems/): The live systems behind every number quoted on this site, with the architecture that produces them
- [AEGIS](https://hikmahtechnologies.com/aegis/): AI systems built around the way you work, not the other way round: the private systems I run my own business on (bookkeeping, infrastructure alerts, mail, research, publishing, analytics), the kit I build client systems with, and when to buy a product instead. Version 1 is open source and archived (49 workflow types over 72 permission-gated tools)
- [AEGIS: how it works](https://hikmahtechnologies.com/aegis/how-it-works/): The permission model, the approval gates and the failure handling of v1, in detail

## Services

- [Data Platform Audit](https://hikmahtechnologies.com/services/data-platform-audit/): In one week: exactly what to fix, what to cut, and what your data platform should cost
- [Data Platform Build](https://hikmahtechnologies.com/services/data-platform-build/): Ingestion to warehouse to dashboards in weeks — for less than a hire's signing bonus
- [AI Automation Build](https://hikmahtechnologies.com/services/ai-automation-build/): LLM workflows in real operations, with a human approval gate

## Products

Hikmah Technologies owns and operates all of these. Each runs its own business on its
own domain; the on-site page explains what it is and what claim it stands as evidence for.

- [aeocue](https://hikmahtechnologies.com/products/aeocue/): Get named when your customers ask an AI — live at https://aeocue.com
- [Ansaar](https://hikmahtechnologies.com/products/ansaar/): AI-powered Shariah-compliant investing platform — live at https://ansaar.in
- [Domain Posture](https://hikmahtechnologies.com/products/domain-posture/): Audit-grade domain hygiene reports — live at https://domainposture.com
- [AuditEase Pro](https://hikmahtechnologies.com/products/auditease-pro/): A white-label internal audit system, deployed for one organisation — live at https://www.auditeasepro.com
- [Quantamental](https://hikmahtechnologies.com/products/quantamental/): Global macro & central-bank credibility analytics — live at https://www.quantamentry.com
- [KoyraCloud](https://hikmahtechnologies.com/products/koyracloud/): Open-source self-hosted PaaS for Docker Swarm — live at https://koyracloud.com

aeocue is sold from this domain: Hikmah Technologies is the seller of record for it
(GSTIN 27AHDPA9121G1ZQ), so its subscription checkout lives here while the brochure
stays on aeocue.com.

- [Start with aeocue](https://hikmahtechnologies.com/aeocue/subscribe/): Subscribe to aeocue: ₹4,999/month founding rate per business location (₹14,999 list), locked six months, cancel any time with one message

## Case studies

- [UK capital-markets research platform](https://hikmahtechnologies.com/case-studies/uk-research-platform/): Core data system for a UK equity-research platform
- [Sehati](https://hikmahtechnologies.com/case-studies/sehati/): An audit platform where every action is traceable
- [VC Solutions](https://hikmahtechnologies.com/case-studies/vc-solutions/): Lead-generation website for an industrial solvents business
- [Manas Realty](https://hikmahtechnologies.com/case-studies/manas-realty/): Marketing website for an established real-estate firm

## Writing

Every post is written from systems I actually run. This blog is the canonical copy —
syndicated versions elsewhere point their canonical tag back here.

- [MCP prompt injection: what a database row can make an agent do](https://hikmahtechnologies.com/blog/mcp-prompt-injection/): MCP prompt injection through data: I planted instructions in a support ticket, served it to Claude through an MCP server 35 times, and logged what happened. What the models did, what the server can prevent (recipients, approvals, logs), and what it cannot.
- [Local-First Analytics for Multi-Tenant AI Agents](https://hikmahtechnologies.com/blog/local-first-analytics-for-multi-tenant-ai-agents/): An agent is not a security boundary. How to separate data isolation, compute isolation and agent reasoning, and why shared Parquet with small per-tenant DuckDB workers is a good shape for multi-tenant agentic analytics.
- [Convert Parquet to Iceberg (or DuckLake) without rewriting the files](https://hikmahtechnologies.com/blog/convert-parquet-to-iceberg/): How to convert Parquet to Iceberg or DuckLake by registering the files you already have: add_files and ducklake_add_data_files on 90 files, what they check, the Hive-folder trap, and why the table may later delete your originals.
- [Prefect vs Dagster: which tier does your pipeline actually need?](https://hikmahtechnologies.com/blog/prefect-vs-dagster/): Prefect vs Dagster, with cron below them and Temporal above: a decision matrix for which orchestration tier a job needs, Prefect 3.8 facts checked in October 2026, and a run of a real Prefect flow.
- [Best database for stock data: what I store NSE prices in, and what broke](https://hikmahtechnologies.com/blog/best-database-for-stock-data/): The best database for stock data depends less on the engine than on how you key, overwrite and deduplicate prices. How I store daily NSE bars in Parquet and ClickHouse, with the ReplacingMergeTree and FINAL problems that ran a 14 GiB server out of memory, and runnable code.
- [MCP vs function calling: what changes, and when a data team needs MCP](https://hikmahtechnologies.com/blog/mcp-vs-function-calling/): MCP vs function calling is not a choice between two ways of calling tools: MCP clients still use function calling underneath. What changes is who owns the tool definitions and where the code runs. Shown with a working server and a production registry served both ways.
- [Iceberg partitioning: hidden partitions vs Hive folders, measured](https://hikmahtechnologies.com/blog/iceberg-partitioning/): Iceberg partitioning on 1.08 million rows: hidden partitioning against Hive-style folders, the cost of partitioning by hour, partition evolution without a rewrite, and the same idea in DuckLake.
- [Trading calendar in Python: what the NSE holiday list and pandas_market_calendars miss](https://hikmahtechnologies.com/blog/nse-trading-calendar-python/): A trading calendar in Python for the NSE, tested against the exchange's own files: where pandas_market_calendars and exchange_calendars disagree with the NSE holiday list, the weekend and Muhurat sessions no list contains, and the 404 that cost my pipeline seven trading days.
- [How to test an MCP server: MCP Inspector, curl and pytest](https://hikmahtechnologies.com/blog/how-to-test-mcp-server/): How to test an MCP server locally, three ways I ran against the same FastMCP server: the MCP Inspector CLI, plain curl against the 2026-07-28 protocol, and pytest with an in-memory client. Plus the stdout bug that corrupts stdio servers, and how I debug tool calls in production.
- [Iceberg table maintenance: compaction, expire snapshots and orphan files](https://hikmahtechnologies.com/blog/iceberg-table-maintenance/): Iceberg table maintenance, measured on one table: what compaction, expire_snapshots and remove_orphan_files each removed, the defaults that made two of them do nothing, and what PyIceberg can and cannot do yet.
- [NSE bhavcopy in Python: download it, parse it, and check the date inside](https://hikmahtechnologies.com/blog/nse-bhavcopy-python/): How to download and parse the NSE bhavcopy in Python with Polars: which of NSE's three daily files to use, the space that nulled a column on every row, the holiday request that returns the previous day's file, and how to get the ISIN and the historical archive.
- [Is Dagster free? Self-hosting Dagster vs Dagster+](https://hikmahtechnologies.com/blog/is-dagster-free-self-hosting/): Is Dagster free? The open-source core is, under Apache 2.0. What Dagster+ costs (checked 1 October 2026), what self-hosting Dagster costs you in ops time, and a minimal Docker Compose setup.
- [AEGIS v1 is archived. Here is what replaces it](https://hikmahtechnologies.com/blog/aegis-v1-is-archived/): I'm archiving the open-source AEGIS repo. Version 2 splits one system that did everything into separate products on a shared SDK, each with its own database and deploy. Why, what is live, and what happens to the old code.
- [Querying Parquet on S3 with DuckDB: secrets, speed and the settings that matter](https://hikmahtechnologies.com/blog/duckdb-s3-parquet/): How to query Parquet on S3 with DuckDB: credentials with CREATE SECRET, why remote reads get slow, the row group and file sizes that fix it, and where Iceberg support stands.
- [Dagster and dbt: do you need both?](https://hikmahtechnologies.com/blog/dagster-with-dbt/): Dagster with dbt: what each tool does, how dagster-dbt runs dbt models as assets and dbt tests as asset checks, how partitions work, and when Dagster alone is enough.
- [DuckDB vs Spark: which workloads actually need a cluster](https://hikmahtechnologies.com/blog/duckdb-vs-spark/): DuckDB vs Spark comes down to one number: the bytes a job touches, set against what one machine can hold. The sizing math, where a Spark cluster earns its cost, and where it doesn't.
- [Dagster vs Airflow: which to choose for a small data team](https://hikmahtechnologies.com/blog/dagster-vs-airflow/): Dagster vs Airflow for a team of one to five: what each model is good at, what Airflow 3 changed, what each costs, and what still hurt after running a 112-asset Dagster pipeline alone.
- [Iceberg catalogs explained: REST, Glue, S3 Tables, Polaris, Nessie](https://hikmahtechnologies.com/blog/iceberg-catalog-explained/): An Iceberg catalog keeps one pointer per table, to its current metadata file, and swaps it atomically on commit. What that looks like on disk, the catalog types, which Iceberg catalog a small team should pick, and when you need none.
- [MCP security and authentication for data access](https://hikmahtechnologies.com/blog/mcp-security-authentication/): MCP security best practices for servers that touch data: when MCP authentication applies (HTTP, not stdio), how MCP OAuth works under the 2026-07-28 spec, the token passthrough ban, CIMD versus dynamic registration, and the scoped tokens and approval gate I use.
- [Postgres and Iceberg: pg_lake, pg_duckdb and the Postgres lakehouse](https://hikmahtechnologies.com/blog/postgres-iceberg-pg-lake/): Postgres and Iceberg meet in four ways: query the lake, write Iceberg tables, act as the catalog, or replicate into Iceberg. What pg_lake, pg_duckdb and the managed options each do, which are dormant, and a hands-on test of Postgres as the catalog.
- [Build a safe MCP server in Python with FastMCP](https://hikmahtechnologies.com/blog/build-mcp-server-python-fastmcp/): How to build an MCP server in Python with FastMCP 4: a runnable read-only DuckDB tool with a row cap, a timeout that really stops the query, and a call log, plus test output. Then the production version: how my own MCP server gates every tool.
- [Iceberg vs Parquet: do you need a table format?](https://hikmahtechnologies.com/blog/iceberg-vs-parquet/): Apache Iceberg vs Parquet is not a choice between two file formats. Iceberg is a table format whose data files are Parquet. What it adds, what it costs in files and commits, and when plain Parquet is enough, with numbers from a laptop test.
- [Paper Before Real Money: How My Finance Agent Earns the Right to Trade](https://hikmahtechnologies.com/blog/paper-before-real-money/): Maou, the finance agent in my AEGIS platform, now keeps my books and runs a paper trading desk on real prices. Here is how the desk works, the rules that make a paper result mean something, and what has to be true before it touches real money.
- [Data quality checks that catch real bugs, with the SQL](https://hikmahtechnologies.com/blog/data-quality-checks/): The data quality checks worth writing, as runnable SQL: nulls, duplicates, ranges, relationships, freshness, volume, jumps and gaps. How to automate them in a pipeline, how the data quality tools compare, and which checks caught real bugs in mine.
- [The agent writes the code. I'm paid for the rest.](https://hikmahtechnologies.com/blog/the-agent-writes-the-code-im-paid-for-the-rest/): I barely type code any more. My hours go on finding out what a client means, deciding what correct looks like and checking the result against reality. That work is the job now, and it is what you should be paying for.
- [Self-hosted LLM for a small team: what it takes, and when it pays](https://hikmahtechnologies.com/blog/self-hosted-llm-for-a-small-team/): A self-hosted LLM keeps prompts and documents on your own hardware. What it needs in VRAM, how to serve it with Ollama, llama.cpp or vLLM, whether local RAG works, and the job I moved off my own GPU.
- [DuckDB MCP server: MotherDuck's, the extension, or your own](https://hikmahtechnologies.com/blog/duckdb-mcp-server/): Which DuckDB MCP server to run: MotherDuck's local server, its remote MotherDuck MCP, the duckdb_mcp extension, or your own. I sent the same five probes to two of them, covering writes, file reads, row caps and timeouts. Here is what got through.
- [Snowflake warehouse sizes and what each one costs in credits](https://hikmahtechnologies.com/blog/snowflake-warehouse-sizes-and-credit-costs/): Snowflake warehouse size sets the credits you burn per hour: X-Small is 1, and each size up doubles it. The full table, what a credit costs by edition, worked monthly figures, and how to pick a size.
- [Lakehouse vs data warehouse: what a small team actually needs](https://hikmahtechnologies.com/blog/lakehouse-vs-data-warehouse/): Lakehouse vs data warehouse for a small team: what each one is, the data lakehouse architecture in four parts, and the minimum DuckDB lakehouse on a laptop and a bucket, with measured load and query times.
- [Postgres MCP server: which one to run, and how to make it read-only](https://hikmahtechnologies.com/blog/postgres-mcp-server/): A Postgres MCP server lets Claude Code or Cursor query your database. The official one is archived and had a read-only bypass; here is what to run instead, and the database settings that make it safe.
- [dbt with DuckDB: a warehouse that costs nothing](https://hikmahtechnologies.com/blog/dbt-with-duckdb/): dbt-duckdb pairs DuckDB's engine with dbt's models, tests and lineage. The project setup that works, run end to end, plus the limits worth knowing first.
- [dbt MCP server: what to let an agent do with it](https://hikmahtechnologies.com/blog/dbt-mcp-server/): dbt Labs ships an Apache 2.0 MCP server for dbt. Setup takes ten minutes; the real decision is which tool groups you leave switched off.
- [Why AEGIS moved off my local GPU — and still runs on yours](https://hikmahtechnologies.com/blog/why-aegis-moved-off-my-local-gpu/): I ran my personal AI system's thinking on an 8GB GPU at home for months, then moved it to Bedrock. Not because local models are bad — because my GPU is small and the work is bursty. Here are the production numbers, and why the embeddings stayed home.
- [dbt Semantic Layer: what it is and who needs one](https://hikmahtechnologies.com/blog/dbt-semantic-layer-explained/): Define a metric once in YAML and MetricFlow compiles it to SQL. Free to define under dbt Core; the APIs and BI integrations need a paid dbt plan.
- [Your Alerts Have No Identity](https://hikmahtechnologies.com/blog/your-alerts-have-no-identity/): Eight open tasks for one failing probe, and thirteen dedupe schemes that disagreed. How I rebuilt AEGIS's alerting so a problem has a record of its own before any ticket exists — and the three things production changed about it afterwards.
- [A Pile of Documents Is Not Knowledge](https://hikmahtechnologies.com/blog/a-pile-of-documents-is-not-knowledge/): My research agent had indexed 20,400 documents. Prompts had used 78 of 10,284 PDFs. Here's what measuring retrieval instead of ingestion changed — and why the fix wasn't a better filter, it was storing less.
- [A Ledger Is Not a Database Table](https://hikmahtechnologies.com/blog/a-ledger-is-not-a-database-table/): My finance agent read 200 characters of each email and called it accounting. Rebuilding it on a plain-text double-entry journal, with Postgres demoted to an index, fixed the numbers, cut the lane's model calls by 71% and caught a bug that kept paid bills open forever.
- [SQLMesh vs dbt: what it fixes, and what it costs](https://hikmahtechnologies.com/blog/sqlmesh-vs-dbt/): dbt is the default and the ecosystem. SQLMesh is better engineering for dev environments, incremental state and real SQL parsing. Who should actually switch.
- [Polars vs Pandas: the three differences that matter](https://hikmahtechnologies.com/blog/polars-vs-pandas-three-differences/): Lazy execution, no hidden copies, and parallelism by default. What each one changes in practice, why the speed claims are softer than the marketing, and when DuckDB is the better answer than either.
- [Parquet vs CSV: what you actually save, and why](https://hikmahtechnologies.com/blog/parquet-vs-csv-what-you-actually-save/): Parquet is smaller and faster than CSV for three mechanical reasons, not because it is magic. What the compression ratio depends on, why the speed gap collapses on a warm cache, and when CSV is still the right answer.
- [DuckDB vs ClickHouse: library or server? How to choose](https://hikmahtechnologies.com/blog/duckdb-vs-clickhouse/): DuckDB runs inside one program on batch data. ClickHouse is a server for many clients querying fresh data. How to choose, from running both, and why teams use both.
- [dbt Core vs dbt Cloud: what you actually pay for](https://hikmahtechnologies.com/blog/dbt-core-vs-dbt-cloud/): dbt Core is free and does the transformation. dbt Cloud sells scheduling, CI, an IDE, docs and governance — so compare it with Core plus what you self-host.
- [Look-ahead bias in backtesting: five leaks I found in my own pipeline](https://hikmahtechnologies.com/blog/look-ahead-bias-point-in-time-data/): Look-ahead bias in backtesting, from five real leaks in a production trading pipeline: fundamentals joined 45 days early, a validation split by alphabet, a live bar saved as a close. How to avoid look-ahead bias with point-in-time data and an as-of join, with runnable DuckDB and Polars code.
- [Data warehouse consulting: what you are buying](https://hikmahtechnologies.com/blog/data-warehouse-consulting-what-you-are-buying/): A buyer's guide from someone who sells it: the three engagement shapes, what each should cost and deliver, the questions to ask, and when to hire nobody.
- [DuckLake with Postgres and S3: a hands-on setup](https://hikmahtechnologies.com/blog/ducklake-postgres-s3-setup/): Set up DuckLake with a Postgres catalog and S3 object storage: the secrets, the ATTACH, what lands in Postgres and what lands in the bucket, how inlining avoids small files, and what two writers did. Every command run on DuckDB 1.5.6.
- [Adjusted close vs close: the formula, and why I stopped using Yahoo's](https://hikmahtechnologies.com/blog/adjusted-close-vs-close/): Adjusted close vs close, with the adjusted close price formula for splits, bonuses and rights issues, a runnable DuckDB and Polars example, what yfinance auto_adjust does, and the 1,499 fake price jumps that mixing the two caused in my pipeline.
- [Market regime detection in production: what the model actually changes](https://hikmahtechnologies.com/blog/market-regime-detection-in-production/): A three-state hidden Markov model runs daily inside my trading system. Three features, four indices, a monthly retrain, and one rule that decides whether the regime is allowed to touch a position at all. This is what it does, and what it is not allowed to do.
- [DuckDB as an LLM cache: what to cache, what to key on, and where it breaks](https://hikmahtechnologies.com/blog/duckdb-as-an-llm-cache/): DuckDB works as an LLM response cache when one process owns it, and the SQL over the cache is the reason to pick it. The three kinds of cache, the single-writer rule, the key that stops you serving a retired model's answers, and measured numbers for exact, brute-force and HNSW lookups.
- [DuckLake vs Iceberg: which table format for a small team](https://hikmahtechnologies.com/blog/ducklake-vs-iceberg/): DuckLake vs Iceberg, measured: the same 50-commit workload on both, with file counts, metadata size and commit times. What DuckLake is, whether it is production ready, and when Iceberg is still the right call.
- [Survivorship bias in trading: how my backtest only saw the winners](https://hikmahtechnologies.com/blog/survivorship-bias-in-backtesting/): Survivorship bias in trading, measured on a real NSE pipeline: a universe built from today's stock list missed about 115 delisted names. How to get survivorship-bias-free stock data from the exchange's own files, and how to handle delisted stocks in a backtest.
- [DuckDB vs Polars: SQL engine or DataFrame library](https://hikmahtechnologies.com/blog/duckdb-vs-polars/): They are in the same speed class, so speed should not decide it. The real split is SQL and a database file against Python expressions inside your pipeline.
- [DuckDB vs Postgres: when to add DuckDB, and when not to](https://hikmahtechnologies.com/blog/duckdb-vs-postgres/): Keep Postgres as the system of record. Add DuckDB when wide analytical scans slow your app down. A side-by-side table and three ways to run both together.
- [ClickHouse vs Postgres: when to add an analytics database](https://hikmahtechnologies.com/blog/clickhouse-vs-postgres/): Postgres is the system of record. ClickHouse is the analytics engine. When Postgres is still enough, what bites when you move, and how to run both together.
- [DuckDB vs SQLite: when to use each, and how to use both](https://hikmahtechnologies.com/blog/duckdb-vs-sqlite/): Use SQLite for an app's own reads and writes. Use DuckDB for analytics over many rows. A side-by-side table, and how DuckDB queries a SQLite file in two lines.
- [Running DuckDB on your own infrastructure: a production setup](https://hikmahtechnologies.com/blog/running-duckdb-on-your-own-infra/): How to set up DuckDB on a server you control for faster queries — memory and temp-dir settings, Parquet layout, the read-only fan-out pattern, containers, and the mistakes that make a single node look slow.
- [Is DuckDB safe for production? The honest limitations](https://hikmahtechnologies.com/blog/is-duckdb-safe-for-production/): DuckDB is production-ready for a specific shape of workload and genuinely unsafe for another. The single-writer model, durability semantics, memory behaviour and version compatibility — what actually bites, and how to design around it.
- [Can DuckDB be your SaaS product's warehouse? Where the ceiling actually is](https://hikmahtechnologies.com/blog/duckdb-as-a-saas-warehouse/): For a small SaaS, a single-node engine over Parquet handles more than founders expect. The three questions that decide whether it fits, what customer-facing analytics needs, and the specific point where you outgrow it.
- [What a data pipeline actually costs: build, run, maintain](https://hikmahtechnologies.com/blog/what-it-costs-to-build-a-data-pipeline/): What drives the build cost of a data pipeline, what it costs to run and maintain afterwards, how managed-ETL pricing models compare with building your own, and the shapes I see by funding stage.
- [What data engineering actually costs, and how I price it](https://hikmahtechnologies.com/blog/what-data-engineering-costs-and-how-i-price/): The full rate card — audit, build, AI workflow, fractional — with the numbers in USD, plus why builds are scoped ranges instead of a day rate.
- [DuckDB vs Snowflake: when one machine is enough](https://hikmahtechnologies.com/blog/duckdb-vs-snowflake/): DuckDB wins when the data a query touches fits on one machine and few people query at once. Snowflake wins on concurrency and governance. Four questions decide it.
- [Data engineering consultant vs. full-time hire: an honest breakdown](https://hikmahtechnologies.com/blog/data-engineering-consultant-vs-full-time-hire/): The real fully-loaded cost of a full-time data engineer, when a consultant is the better call, and how to tell which one your situation actually needs.
- [How to connect an LLM to your database, safely](https://hikmahtechnologies.com/blog/connect-llm-to-database-safely/): The four ways to wire an LLM to your data — MCP, text-to-SQL, a semantic layer, function calling — and the guardrails that stop the demo becoming an incident.
- [ClickHouse vs Snowflake: which costs less, and when](https://hikmahtechnologies.com/blog/clickhouse-vs-snowflake/): ClickHouse wins on steady, sub-second dashboards if you run it yourself. Snowflake wins on ad-hoc SQL with nothing to operate. How the two cost models differ.
- [Chat with your database: a practical guide, not a product pitch](https://hikmahtechnologies.com/blog/chat-with-your-database/): What it takes to make natural-language querying actually useful for non-technical teams — the schema semantics, guardrails and verification the demos skip.
- [How I built a paid-grade data product on 100% free public data](https://hikmahtechnologies.com/blog/paid-grade-data-product-on-free-public-data/): A teardown of a macro data product: 171 countries scored daily from 10 free public data sources, point-in-time correct, with the licensing and cost work done.
- [LLM agents in production: a teardown of AEGIS, my open-source automation layer](https://hikmahtechnologies.com/blog/llm-agents-in-production-aegis-teardown/): Three LLM agents, 49 Temporal workflows, every risky action behind a human approval gate. An architecture teardown of AEGIS v1, now open source and archived — and what it teaches about production AI automation.
- [Building a data platform solo: the Ansaar teardown](https://hikmahtechnologies.com/blog/building-a-data-platform-solo-ansaar-teardown/): NSE and crypto ingestion, ClickHouse, LightGBM, backtests and a Prolog compliance layer — a self-hosted data platform built and run by one person.
- [AEGIS Is Open Source](https://hikmahtechnologies.com/blog/aegis-is-open-source/): Three months ago I wrote about my personal AI orchestration system and ended with a section titled 'Why It's Not Open Source Yet.' That's fixed. AEGIS is on GitHub, MIT-licensed — here's what it actually took to get there.
- [Behavior Is Data, Not Code](https://hikmahtechnologies.com/blog/behavior-is-data-not-code/): The biggest refactor of the AEGIS open-sourcing sprint: removing every branch on an agent's identity so behavior lives in the database as capability tags — resolved at runtime, edited from a UI, and safe even when a tag has no owner.
- [A To-Do Is a Tweet: Social Publishing With Approval Cards](https://hikmahtechnologies.com/blog/a-todo-is-a-tweet/): Scheduling a social post doesn't need a new UI. A to-do already has copy, a time, and labels — so in AEGIS, a Todoist task with a publish label is a scheduled post, and nothing goes out until a card in chat gets a human tap.
- [Bring Your Own Cloud: The Infrastructure Registry](https://hikmahtechnologies.com/blog/bring-your-own-cloud-infrastructure-registry/): For a self-hosted system that's meant to be forked, there's exactly one honest way to handle infrastructure credentials: the user brings their own, the system stores them encrypted, and nothing in the code assumes a vendor. Here's how AEGIS got there — and what it cost me to cut my own vendors out.
- [Labels, Not Projects: Rethinking GTD](https://hikmahtechnologies.com/blog/labels-not-projects-rethinking-gtd/): The day Next and Someday stopped being Todoist projects and became labels — and why that small modelling decision is what made AEGIS's GTD layer, and its bidirectional sync, actually work.
- [Day One of Opening the Box](https://hikmahtechnologies.com/blog/day-one-of-opening-the-box/): AEGIS is going open source, and today the first public commits landed. Day one of turning a private system into a shippable one: twenty-seven migrations squashed into a baseline, credentials evicted from the build, and an admin panel redesigned around decisions.
- [Your Snowflake bill is mostly overhead](https://hikmahtechnologies.com/blog/your-snowflake-bill-is-mostly-overhead/): If you're paying thousands a month to query tens of gigabytes, you're renting a freight train to carry a backpack. Here's how to tell, and what to do instead.
- [When the Bot Learns to Stay Quiet](https://hikmahtechnologies.com/blog/when-the-bot-learns-to-stay-quiet/): AEGIS watches my homelab and GitHub, but the interesting part is what happens between an alert firing and me hearing about it. Most of the time, the correct answer is nothing.
- [DuckDB in production: what it's actually good at (and what it isn't)](https://hikmahtechnologies.com/blog/duckdb-in-production-what-its-good-at/): Is DuckDB production-ready? Where it shines, the workloads it fits, and the deployment patterns that work — from someone who ships it.
- [Do you actually need a data warehouse?](https://hikmahtechnologies.com/blog/do-you-actually-need-a-data-warehouse/): A short, honest decision framework for the question every data team eventually faces — before you sign a five-figure annual contract for infrastructure your data might not need.
- [Ask your database in plain English — locally](https://hikmahtechnologies.com/blog/ask-your-database-in-plain-english-locally/): A small SQL-tuned model running on your machine can convert plain English questions into correct SQL. No API keys, no data leaving your laptop, no vector database.
- [Local-first analytics in practice: DuckDB, Parquet, and killing the round-trip](https://hikmahtechnologies.com/blog/local-first-analytics-in-practice-duckdb-parquet/): A concrete walkthrough of the local-first pattern: query Parquet directly with DuckDB, no warehouse, no server, no per-query bill — from a notebook or the browser.
- [Calling an LLM from inside a SQL query](https://hikmahtechnologies.com/blog/calling-an-llm-from-inside-sql/): Run a model directly in SQL to classify rows, extract text fields, or summarize data without exporting. Here's where it works, what it costs, and when not to bother.
- [Why I wrote Local-First Analytics](https://hikmahtechnologies.com/blog/why-i-wrote-local-first-analytics/): Most analytics stacks are cloud round-trips solving problems that fit on a laptop. Local-first analytics is the case for bringing the compute back to the data — and to the user.
- [Turning messy PDFs into clean tables, on your own machine](https://hikmahtechnologies.com/blog/messy-pdfs-to-clean-tables-locally/): Parse PDFs to markdown, extract structured data with a local model and strict validation schema, no API needed.
- [Connecting an AI assistant to your database with MCP — without letting it do damage](https://hikmahtechnologies.com/blog/connect-an-ai-assistant-to-your-database-with-mcp/): Use the Model Context Protocol to let Claude explore your data in plain English. The whole game is permissions: read-only role, standard server, 90% of the value with almost none of the risk.
- [Search your documents without a vector database](https://hikmahtechnologies.com/blog/search-your-documents-without-a-vector-database/): Most teams reach for Pinecone or Weaviate the moment they hear semantic search. For a team-sized knowledge base, you don't need one. Your database already has what you need.
- [Letting Claude Code write your dbt models — and why you still read every line](https://hikmahtechnologies.com/blog/letting-claude-code-write-your-dbt-models/): Claude Code can run dbt in a loop, read errors, and fix them. It needs you to steer grain, business logic, and naming — and to review the tests it writes.
- [Using an LLM to catch the data quality problems your tests can't](https://hikmahtechnologies.com/blog/llm-as-judge-for-data-quality/): Structural tests catch NOT NULLs and schema errors. Semantic problems—does this description match its category?—need a judge. How to use one responsibly.
- [Should there be an LLM in your data pipeline?](https://hikmahtechnologies.com/blog/should-there-be-an-llm-in-your-pipeline/): An LLM is probabilistic, slow, and costly. Here's how to decide whether it belongs in your pipeline — and where it clearly doesn't.
- [Local model or paid API? The honest math for data work](https://hikmahtechnologies.com/blog/local-model-or-paid-api-the-honest-math/): When does running an open-weight model on your own hardware beat paying per token? The answer depends on volume, frequency, and whether you have real privacy constraints.
- [Generating realistic test data with a local model](https://hikmahtechnologies.com/blog/generating-test-data-with-a-local-model/): How to harden a data pipeline by feeding it carefully generated edge cases, using a local LLM when simpler generators aren't enough.
- [When local-first runs out of road](https://hikmahtechnologies.com/blog/when-local-first-runs-out-of-road/): Local-first, file-based data has a real ceiling. Most teams never reach it. When you do, here's what actually changes.
- [Running a local model over millions of rows without it falling over](https://hikmahtechnologies.com/blog/running-a-local-model-over-millions-of-rows/): Batching, checkpointing, and idempotency are what make local inference scale. Get these right and a single GPU chews through millions of rows overnight.
- [One Primitive for Every Interruption](https://hikmahtechnologies.com/blog/one-primitive-every-interruption/): The design decision in AEGIS I'm proudest of isn't an agent or a model — it's a single interactions primitive. How one Postgres table, five kinds of card, and a Temporal workflow replaced every per-domain approval pattern.
- [Meet AEGIS: My Weird Little Operating Layer](https://hikmahtechnologies.com/blog/meet-aegis-my-weird-little-operating-layer-3087a3e616d4/): AEGIS is my personal operating layer: four agents that watch my inbox, repos, alerts and feeds, and bring a proposed next step instead of a notification.
- [Building with Claude Code: Faster Iteration, Faster Drift](https://hikmahtechnologies.com/blog/building-with-claude-code-faster-iteration-faster-drift-480a646e6f3c/): Claude Code made my AEGIS v3 rewrite fast. The honest part is what got harder: drift, verification and knowing when to stop. The developer stays hands-on.
- [Building AEGIS: How Prolog + a Knowledge Graph Make Local LLMs Actually Useful](https://hikmahtechnologies.com/blog/building-aegis-how-prolog-a-knowledge-graph-make-local-llms-actually-useful-5cc5217783d2/): Prolog and a knowledge graph take routing, classification and fact lookup off the LLM, so most of a personal AI system runs on a cheap local model.
- [Regime Detection with Hidden Markov Models in Python](https://hikmahtechnologies.com/blog/market-regime-detection-from-hidden-markov-models-to-wasserstein-clustering-6ba0a09559dc/): Fit a 2-state hidden Markov model (hmmlearn GaussianHMM) to S&P 500 returns to find market regimes, then compare it with Wasserstein clustering. Python code for both.

## Contact

- Email: info@hikmahtech.in
- Book a 30-minute call: https://calendly.com/arshad-hikmahtechnologies/30min
- Location: Navi Mumbai, India — working with clients in the US, EU and India