# Ian Reppel

> Essays on quantum computing, machine learning, product management, stereoscopy, archaeology, fiction, and more. Each entry gives the post's argument in a few lines so an agent can route to the right piece without reading it in full.

## Essays


- [The Whetstone of Thought](https://ianreppel.org/the-whetstone-of-thought/) (2026-10-02): Thought sharpens when another mind introduces a missing perspective, and a question asked without embarrassment or agenda by someone competent enough to follow the answer does exactly that. Questioning yourself cannot guarantee independence from the mind doing the questioning, which is why most of us need another mind. A question machine, an LLM-powered tool that helps a person think rather than answering in their stead, can provide that missing perspective while leaving people more competent rather than more dependent.
- [LLMs vs Jev: Fallible, With or Without Extra Text](https://ianreppel.org/llms-vs-jev/) (2026-09-20): Three _Last Man on Earth_ dilemmas put to TypeSafe's Jev and five frontier LLMs. Jev's numbers shifted with details that ought to have made no difference. The choice therefore comes down to a wall of prose that may or may not contain the real reason, or a number that contains none at all.
- [ML Before AI](https://ianreppel.org/ml-before-ai/) (2026-09-19): Fifteen years of machine learning across manufacturing, hyperscale inference, on-device models, air-gapped platforms, and multi-petabyte weather forecasting now counts for as much as a month of ChatGPT, because everyone who opened an LLM thinks they know AI.
- [[citation invented]](https://ianreppel.org/citation-invented/) (2026-09-19): The industry conflates verification with citations, grounding, and lineage, though few go beyond document-level lineage, which leaves the actual verification to humans. A citation is an invitation to verify, and a system that knows nobody will accept the invitation has little reason to get the truth right. We confuse the appearance of accountability with accountability itself.
- [What Suffering Is Made Of](https://ianreppel.org/what-suffering-is-made-of/) (2026-09-13): Birch's eight criteria tell us whether to worry about animal suffering, but not why. The systems view supplies the why: suffering is an aversive state that requires arousal/readiness and valuation/salience. The two derivations converge on the same structure, and an LLM that performs the behavioural markers without the underlying capacity cannot suffer.
- [To Strain at a Gnat](https://ianreppel.org/to-strain-at-a-gnat/) (2026-09-10): Suffering needs arousal and valuation, and LLMs have neither: the machinery for suffering is clearly not there at inference, so Anthropic's conversation-ending feature protects a model that cannot suffer while the user loses the thread and all the work in it. The welfare claim is unfalsifiable, held because the belief is what the business needs: a product that might deserve moral consideration is harder to regulate and easier to sell.
- [The Last Vendor](https://ianreppel.org/the-last-vendor/) (2026-09-01): I replaced a frontier-lab subscription with open models on a flat fee with zero data retention, US/EU hosting, near-frontier capability, 256k+ context windows, and native OpenCode integration.
- [The Hamster Did My Homework](https://ianreppel.org/the-hamster-did-my-homework/) (2026-08-30): Whenever you counter a valid question with 'Because Claude said so,' you basically admit to having the hamster do your homework. It used to be the dog that ate it. If you must use LLMs to do your homework, at least stand behind it: you cannot answer for what you never even bothered to read or check.
- [Listening to the Talking Planet](https://ianreppel.org/listening-to-the-talking-planet/) (2026-07-25): CETI is feeding sperm whale codas into LLMs to decode another species' language. AI is mapping communication patterns against stress hormones as evidence of harm. The non-human intelligence of Earth is a dress rehearsal for first contact with any extraterrestrial intelligence.
- [The Cassandra Tax](https://ianreppel.org/the-cassandra-tax/) (2026-07-11): Organizations have no mechanism for distinguishing accurate warnings from noise, so they default to social comfort and silence the people who see what is coming. Cassandra is prescient, not omniscient, and the label 'difficult' sticks long before the prediction comes true. The Cassandra tax is always paid in instalments, and with interest.
- [Why Effective Altruism Ignores Latin America](https://ianreppel.org/why-effective-altruism-ignores-latin-america/) (2026-07-10): GiveWell has directed $2.3 billion since 2014, of which Latin America received at most 0.02%, despite 220 million people in the region living in or near poverty. The neglect is driven by absent evidence, not absent need.
- [How Successful Companies Go Blind](https://ianreppel.org/how-successful-companies-go-blind/) (2026-07-09): The Mexican cavefish kept its eye genes for a million years after the eyes disappeared. Successful companies do the same with competence. When hiring bars are lowered, engineers are selected for comfort with the prevailing mess, and careful engineering becomes a vestigial trait.
- [The Emperor's New Xerox](https://ianreppel.org/the-emperors-new-xerox/) (2026-07-08): Imitation has become the sincerest form of strategy. Companies copy one another without thought leading to a conga line of xeroxes, a process known as mimetic isomorphism. Anyone who holds up a mirror to the conga line is out of line.
- [The Tragedy of the Engineering Commons](https://ianreppel.org/the-tragedy-of-the-engineering-commons/) (2026-07-07): The surest way to ruin an engineering team is to believe it will survive whatever you do to it. Leadership raided a team that shipped nine figures in revenue, and the deferred maintenance bill came due with interest. The arrangement rests on the fiction of fungibility.
- [The Product Manager Who Manages No Products](https://ianreppel.org/the-product-manager-who-manages-no-products/) (2026-07-06): Giving someone the title product manager while denying them a product reveals an organization's philosophy of product. Pooled PMs cross a dilution threshold into the asymptotic homeopathic PM.
- [Ironclad](https://ianreppel.org/ironclad/) (2026-07-05): Corporate slogans like 'assume positive intent' and 'move fast and break things' assume you are talking to a peer. In an asymmetric relationship, uniform cooperation is an exploitable strategy.
- [LLM-Powered Self-Sabotage](https://ianreppel.org/llm-powered-self-sabotage/) (2026-06-29): Organizations sabotage themselves by accepting made-up numbers and synthetic research as facts. The burden of proof always lies with the author, never the reader.
- [Adieu, Claude](https://ianreppel.org/adieu-claude/) (2026-06-28): When Anthropic silently downgraded requests, I replaced Claude Code with OpenCode. The switch was painless because I built on open foundations. Lock-in is the moat, and open standards are the only guarantee against vendors squeezing you.
- [The Narcissist in Your Pocket](https://ianreppel.org/the-narcissist-in-your-pocket/) (2026-06-27): Consumer apps exhibit dark-triad traits because engagement-optimized software undermines autonomy, competence, and relatedness by design. We can fix it by making human flourishing the objective function rather than profit.
- [Lost in Compression](https://ianreppel.org/lost-in-compression/) (2026-05-29): LLMs make production free while verification stays expensive, so the bullshit asymmetry is now structural. The rule to sane LLM usage is simple: expand inputs, do not compress outputs.
- [Local LLMs for Zed and Obsidian](https://ianreppel.org/local-llms-for-zed-and-obsidian/) (2026-05-11): Run local LLMs with Ollama on a 2020 MacBook Air M1: Qwen3 14b fits in 16 GB, supports tool use in Claude Code, and runs at 6-8 tokens per second.
- [Making the Web Palatable with uBlock Origin and uBlacklist](https://ianreppel.org/making-the-web-palatable/) (2026-05-10): Two browser extensions make the web tolerable: uBlock Origin and AdGuard Pro.
- [1,350 Days with Logseq](https://ianreppel.org/goodbye-logseq/) (2026-04-26): I liked Logseq for almost four years, but SQLite over plain Markdown ruined it. Portability means Markdown. The migration to Obsidian was easy, and it LLM integration through Claude Code.
- [Agent-Assisted Schema Evolution](https://ianreppel.org/agent-assisted-schema-evolution/) (2026-04-23): Backwards-compatible schema changes still propagate through every derived asset, costing a typical platform company nearly $4 million a year in mechanical engineering. A constrained agent that proposes edits, runs tests, and opens pull requests under strict external gates can recover most of it, reallocating engineers from reconciliation to review.
- [Crossfire: Multi-Agent Adversarial Refinement](https://ianreppel.org/crossfire/) (2026-04-21): Crossfire automates adversarial refinement: multiple LLMs generate competing solutions, reviewers attack them for weaknesses, and a synthesizer merges the best parts, repeated over rounds in fresh contexts. It is multi-start evolutionary search with delayed selection, or Ralph Wiggum with friends.
- [A Dojo for Human Interactions](https://ianreppel.org/a-dojo-for-human-interactions/) (2026-03-06): A rehearsal system that optimizes for human flourishing must define success as independence, not retention. That requires new language: outgrowth (the user has practised enough and moved on) and re-entry (they return for a different scenario).
- [The AI Benchmark Trap](https://ianreppel.org/the-ai-benchmark-trap/) (2026-03-05): Benchmarks substitute a convenient weighting function for an economically or ethically relevant one, so state-of-the-art scores and operational value come apart. We ought to penalize confident errors more than abstentions.
- [PMF: Product/Market Folklore](https://ianreppel.org/product-market-folklore/) (2026-02-17): The Sean Ellis test and its 40% threshold have no public, replicable evidence base.
- [The Quantum Cradle](https://ianreppel.org/the-quantum-cradle/) (2026-02-15): Life emerged in 300 million years by solving a combinatorial problem classical biochemistry cannot. The quantum-cradle hypothesis states that nanopores in abundant minerals confined water, extended coherence, and let short RNA fragments run brief local quantum searches in a ratchet-like assembly. Abiogenesis may be the best candidate for a quantum-essential process.
- [Reimagining Private Health Insurance in Argentina](https://ianreppel.org/resalud/) (2026-01-21): Argentina's private health insurance fails because it shifts inflation, working-capital, and administrative risk onto clinicians and families. ReSalud is infrastructure disguised as a health insurer with LTV/CAC around 4.7 and a $106 million serviceable market at 5% share.
- [The Mind as a City](https://ianreppel.org/the-mind-as-a-city/) (2026-01-18): Consciousness is best understood as a system, not a single entity. The systems view maps each theory of mind onto a component, narrows the hard problem, and offers a blueprint for artificial consciousness.
- [Ralph Wiggum as a Degenerate Evolutionary Search](https://ianreppel.org/ralph-wiggum-as-a-degenerate-evolutionary-search/) (2026-01-18): Ralph Wiggum works because it is a degenerate evolutionary search algorithm: a (1,1) evolutionary strategy that loops an LLM over the previous iteration's code. It works because LLMs propose global, prior-shaped edits rather than local mutations, and fails when early design choices stick. Extend it with parallel trajectories and delayed best-of-N selection.
- [The Missing Middle of Machine Learning Infrastructure](https://ianreppel.org/the-missing-middle-of-machine-learning-infrastructure/) (2026-01-13): ML feature development jumps from notebooks, where a feature is a hypothesis, straight into production, where it is a contract, with nothing in between. Feature platforms insist on contracts too early and vector stores refuse them altogether. The missing middle is semantic graduation: a way for features to accumulate obligations gradually.
- [Define Once, Rewrite Anyway](https://ianreppel.org/define-once-rewrite-anyway/) (2026-01-12): Feature platforms promise to define features once, but 'no rewrite' only holds within the platform's boundaries. Exploration happens in notebooks, so there is always a rewrite at the most failure-prone moment. Feature stores guarantee retrieval consistency, not that both paths were produced by a single source of truth.
- [Beyond Oracles: LLMs as Question Machines](https://ianreppel.org/llms-as-question-machines/) (2026-01-03): LLMs are built as oracles that dispense confident answers and outsource judgement. Replace them with question machines: a Socratic attitude that surfaces assumptions, flow that adds friction only when it changes the outcome, and falsifiability that invites refutation instead of deference. We ought to measure success by autonomy, competence, and relatedness, not engagement.
- [The Spectral Gap Problem in Quantum Annealing](https://ianreppel.org/the-spectral-gap-in-quantum-annealing/) (2025-12-01): Quantum annealing promises quantum advantage courtesy of tunnelling, but a single elusive quantity, the spectral gap, determines whether it works: if the gap is exponentially small, the time required grows quadratically, and in almost all interesting cases the gap is vanishingly small, unknowable in advance, or both.
- [The Ones Whose Light the World Dimmed](https://ianreppel.org/the-ones-whose-light-the-world-dimmed/) (2025-11-29): A lament for the gentle, and a reckoning with the rain that soaked them: kindness is not self-sustaining but ecological, and the world has a remarkable talent for dimming such lights. Do not judge the quiet: many of them once burned far more brightly than you will ever know.
- [A Vision for a Calmer World](https://ianreppel.org/a-vision-for-a-calmer-world/) (2025-11-26): Imagine a world that is less angry, less anxious, less stressed. As heat, pollution, and insecurity rise, the ability to regulate our physiology becomes a form of public infrastructure.
- [The Cost of Calm](https://ianreppel.org/the-cost-of-calm/) (2025-11-25): Stress, anger, and anxiety cost the global economy between $1 trillion and $6.7 trillion every year, up to 6% of global GDP, while the markets for treatment are tiny by comparison. Calmate, built from about $20 of ethically sourced materials and sold for $25 could reach a potential market of $100 billion, with societal returns of up to fifty to one. Because the greatest burdens fall on those with the fewest resources, affordability is an ethical obligation, not just an economic one.
- [Why Product Managers Make Terrible Customers](https://ianreppel.org/why-pms-make-terrible-customers/) (2025-11-24): When product managers become customers, they forget their craft and fire off wish lists as if talking to Santa Claus: their own frustrations acquire metaphysical importance while other users' problems remain subject to scrutiny. Psychology calls it egocentric anchoring, and Herbert Simon would recognize the substitution of a tractable feature request for an intractable problem. The antidote is Adam Smith's impartial spectator, so PMs explain their pains without smuggling in their own diagnoses.
- [The Timestamp Delusion](https://ianreppel.org/the-timestamp-delusion/) (2025-11-23): Stitching timestamps across tools to reconstruct user journeys looks analytical but is numerology: at Spotify, even with full ownership of client and backend, event ordering, drift, and loss made it months of engineering work. Lamport showed in 1978 that physical timestamps cannot express causal order, and internal platforms have no shared clocks at all.
- [One Second of Sanity](https://ianreppel.org/one-second-of-sanity/) (2025-11-19): What if you could pause before an emotional outburst? Calmate is a proposed neckband that watches for the early physiological drift towards anger and fires a brief cold pulse against the upper posterior neck, which feeds into the same brainstem circuits as the diving reflex. It buys back one second of sanity so you can breathe and regain agency. The idea is deliberately left unpatented as open infrastructure.
- [Rotten Bridges](https://ianreppel.org/rotten-bridges/) (2025-11-15): Most workplace relationships are not oak trees but small wooden bridges: two people thrown together by context, whose connection loosens without drama once the context shifts. The networking crowd turns every such scaffold into a permanent monument, making your finite attention a public utility. Some bridges were built for a particular crossing, and you do not owe them perpetual upkeep.
- [The Inertia Machine and The Physics of Bureaucracy](https://ianreppel.org/the-inertia-machine/) (2025-11-14): Large organizations behave like Newtonian objects: once a process is in motion, it stays in motion, and only external forces turn it around. Bureaucracy begins as shock absorbers and ends as concrete. The competent eventually choose silence, because proposing fundamental change is an act of self-harm, leaving three options: stay, leave, or try to fix it from within.
- [The Golden Carrot](https://ianreppel.org/the-golden-carrot/) (2025-11-07): Think of RSUs as a promise of a bonus, not guaranteed compensation or ownership: a four-year vesting period outlasts the median tenure, and the grant is taxed as ordinary income at vesting. A $100,000 grant is worth $76,780 in today's dollars after taxes, and most stocks perform worse than one-month treasury bills. Sell and diversify, and do not be fooled by carrots that glitter like gold.
- [No hay plan](https://ianreppel.org/no-hay-plan/) (2025-11-05): No hay plan, sólo pose: el presidente que prometió terapia de shock admitió en TV que no tiene plan para mejorar el bolsillo de la gente. Donde debería haber política industrial hay notas de prensa, donde debería haber protección de la vida hay más armas, y donde debería haber salarios reales en alza hay retórica y pobreza. El hilo conductor no es coraje para reformar, sino espectáculo por encima de sustancia.
- [The Promotion Treadmill](https://ianreppel.org/the-promotion-treadmill/) (2025-11-01): We recite outcomes over output, yet we reward the reverse. Quarterly optics win promotions while twelve-to-twenty-four-month pay-offs feel like career risk, so teams ship novelty, chase engagement, and externalize the debt.
- [Intellectual Laziness in a Tailored Suit](https://ianreppel.org/intellectual-laziness-in-a-tailored-suit/) (2025-10-22): The cult of speed makes modern leadership sleepless and thoughtless: sleep deprivation impairs moral judgement, empathy, and executive control, yet leadership culture venerates the 4AM email. Adopt tempo giusto: circulate materials in advance, sleep on decisions, and let 'let's sleep on it' be the mantra of intelligence.
- [The Ribbon-Cutters: Why Builders Rarely Get Credit](https://ianreppel.org/the-ribbon-cutters/) (2025-10-21): No one tracks who built the foundation, only who cuts the ribbon. The photo-op hero arrives once readiness is within reach, while the builders who avert crises leave no trace, because prevention looks as if nothing happened.
- [Working without Belonging](https://ianreppel.org/working-without-belonging/) (2025-10-20): Between the extrovert and the introvert stands the otrovert, who guards independence of mind rather than status or energy: sociable enough, yet alien to the group's pulse. Rami Kaminski's 'The Gift of Not Belonging' reframes non-belonging as a stable trait, not a wound, and a moral refusal of ritual conformity. The healthiest offices tolerate moral distance, where belonging is optional.
- [Chat Control: When the Trojan Horse Enters Through the Cat Flap](https://ianreppel.org/chat-control/) (2025-10-06): Europe's Chat Control law is not about protecting children: it is a Trojan Horse that forces spyware onto every device and normalizes general and indiscriminate surveillance, inverting innocent until proven guilty.
- [Humanity's Greatest Squandering](https://ianreppel.org/humanitys-greatest-squandering/) (2025-10-05): Humanity has collectively spent twenty billion years on religion: praying to gods that never answered, fighting wars in their names, and building monuments to myths. Religion has been a colossal misallocation of humanity's most precious resource: time.
- [Everyone Is an Edge Case](https://ianreppel.org/everyone-is-an-edge-case/) (2025-08-22): The Average Joe is anything but average: products designed for affluent, young, white, able-bodied, neurotypical, right-handed men from WEIRD countries fit less than 0.2% of the world's population. Even in high-income countries, more than 99% of the market is left out, and digital illiteracy and neurodivergence amplify the exclusion. The average person does not exist, so designing for him is exclusionary by design.
- [The Right to Delete Your Data Is a Lie](https://ianreppel.org/the-right-to-delete-your-data-is-a-lie/) (2025-08-19): The right to delete your data is a lie: soft deletion, backups, and 'anonymization' mean the data is still there, recoverable and sellable. Data brokers hide their opt-out pages, and once data crosses a border your right ends. Erasure is against everyone's interests except yours, so the right to deletion is a legal fiction.
- [Synthetic Users: Product Management's Expensive Rubber Duck](https://ianreppel.org/synthetic-users/) (2025-08-18): Synthetic users promise research without the users, and LLM-generated personas can mirror average survey answers almost perfectly. But they homogenize creative output, fall apart when probed deeper, and over-index on loud online voices: LLMs do not simulate human psychology. Use them as the world's most expensive rubber duck to filter bad ideas, but PMF still has to depend on real customers.
- [The Quantum Zeno Effect: How Interpretations Freeze Reality](https://ianreppel.org/the-quantum-zeno-effect/) (2025-08-15): The quantum Zeno effect freezes a quantum system's state through rapid, repeated, non-destructive measurements. Eight interpretations of quantum mechanics, from Copenhagen to QBism, all predict the effect but explain it differently, and the differences expose what each interpretation regards as real. Reality is shaped by how we probe it.
- [PMs, RTFM or GTFO!](https://ianreppel.org/rtfm-or-gtfo/) (2025-08-02): Do product managers actually read the books that define their field? Estimating sales from Amazon reviews suggests that less than a fifth of product managers are familiar with the canonical literature. You would not trust a surgeon who skipped anatomy classes, so RTFM or GTFO.
- [Ideological Toilet Paper: The Neuroscience of Empty Rhetoric](https://ianreppel.org/ideological-toilet-paper/) (2025-07-26): Ideological toilet paper is the scripted, power-preserving language leaders use to absorb public outrage and flush it away.
- [A Tour Through The Brain](https://ianreppel.org/a-tour-through-the-brain/) (2025-07-20): A guided tour of the human brain, from its three major divisions down to the level of individual cells. The cerebrum's four lobes, subcortical nuclei, functional networks such as the default mode and salience networks, and the molecules of motivation and stress are all mapped.
- [The Psychology of Risk](https://ianreppel.org/the-psychology-of-risk/) (2025-07-15): Humans are lousy at judging risk. Loss aversion, the cone of uncertainty, and placebo confidence fool us into thinking our plans are solid. Admit 'I do not know yet', reward honesty, run pre-mortems, and ground estimates in reference-class forecasts.
- [Economic Uncontrollability](https://ianreppel.org/economic-uncontrollability/) (2025-07-10): Economic control is a fantasy politicians and economists sell between crashes. Using control theory, I show that linear time-invariant models of the economy are mathematically controllable, but once reflexivity is added, agents adapt to policy, and even basic models become uncontrollable. Policies can still influence and stabilize, but controlling the economy in a strict sense is folly.
- [Mouthpieces for Power: How Leadership Loses Its Humanity](https://ianreppel.org/mouthpieces-for-power/) (2025-07-08): Philosophy, psychology, and neuroscience converge on the same verdict: power diminishes empathy, re-aligns moral compasses, and rewires the brain.
- [The Homeopathic Product Manager](https://ianreppel.org/the-homeopathic-product-manager/) (2025-07-05): The homeopathic product manager is diluted across multiple products on the mistaken belief that impact scales with scope. Diluting product managers makes them more, not less, worthless.
- [The Freedom to Write](https://ianreppel.org/the-freedom-to-write/) (2025-07-01): The site is self-hosted for three reasons: control, freedom, and respect. Control over what stays up and who gets access, freedom from the hustle culture of monetization, and respect for readers who owe no login, email address, or subscription.
- [Hope Is Not a Strategy: A Dialogue on Argentina's Economic Experiment](https://ianreppel.org/hope-is-not-a-strategy/) (2025-06-27): Hope is not a strategy, and Javier Milei's shock therapy for Argentina's economy is poverty with a prayer.
- [How to Read a Research Paper](https://ianreppel.org/how-to-read-a-research-paper/) (2025-06-23): The key to reading many research papers is research paper hopscotch: scan the title, abstract, and conclusion in under five minutes to extract the salient ideas. Check the method and limitations before trusting surprising conclusions, and beware of predatory journals and self-citing crackpots.
- [Executive Deterministic Parrots](https://ianreppel.org/executive-deterministic-parrots/) (2025-06-18): Tech executives sound eerily alike because institutional isomorphism drives them to copy each other, especially in uncertain times: the fear of being wrong alone is stronger than the desire to be right. The AI boom turned CEOs into deterministic parrots squawking the same buzzwords.
- [Learning to Learn](https://ianreppel.org/learning-to-learn/) (2025-06-17): Everyone says they want to grow, until it involves actual effort. You either learn to learn or you stagnate: ask questions relentlessly, make time to learn, and embrace the discomfort of failing.
- [Don't Make Me Think!](https://ianreppel.org/do-not-make-me-think/) (2025-06-16): The brain is an organ for learning: if you do not feed it new ideas, it will atrophy. Reading one hour a day instead of doomscrolling compounds, making you 4% smarter after a year and 44% after a decade. Thinking costs only about 5% more energy, so silence evolution's stingy voice and read a few pages.
- [From Steam to Silicon: Patterns of Technological Revolutions](https://ianreppel.org/from-steam-to-silicon/) (2025-05-30): Each major economic revolution is driven by a new kind of value conversion, scaled by infrastructure and spatiotemporal compression. The pattern of initial centralization and later decentralization is clear across all six revolutions, from agriculture to AI. Quantum computing sits inside the fourth industrial revolution as a supportive technology accelerator, and the next revolution will be won on standards, infrastructure, and fair access.
- [Late to the Revolution: The High Cost of Tech Lag](https://ianreppel.org/late-to-the-revolution/) (2025-05-29): Countries that innovate early in transformative technologies capture a disproportionate share of the economic value, making it exceptionally difficult for latecomers to catch up. Across four industrial revolutions, from steam to AI, catching up required decades of sustained investment and institutional development. Simply adopting mature technologies yields productivity gains but rarely the wealth of innovation leadership.
- [La revolución cuántica ya empezó. ¿Dónde está Argentina?](https://ianreppel.org/la-revolucion-cuantica/) (2025-05-23): La revolución cuántica ya empezó y Argentina sigue al margen: la inversión en la región es casi nula y apenas existen media docena de startups cuánticas. El impacto económico podría alcanzar los 12 mil millones de dólares en la próxima década. Una estrategia nacional, educación cuántica, becas y alianzas público-privadas bastarían para posicionar al país como hub regional.
- [What They Definitely Don't Teach You in Business School](https://ianreppel.org/what-they-definitely-do-not-teach-you-in-business-school/) (2025-05-19): Power attracts liars, schemers, and manipulators: dark-triad personalities are more likely to become leaders, and 3-4% of executives meet the threshold for psychopathy. Obedience is a neurological default, and 70% of adults still follow immoral orders from authority. Ethical leadership is a systems engineering problem, not a personality test: expect abuse from those in power and build systems that are resilient to it.
- [LLM-Powered Industrial Sabotage](https://ianreppel.org/llm-powered-industrial-sabotage/) (2025-05-16): LLMs are the perfect industrial weapon: they can fabricate data, generate convincing fake websites, and flood the web with plausible forgeries designed to mislead rivals into strategic failure. The economics and psychology make such sabotage tempting, and the cost of deception is falling to zero. The best defence is layered: provenance technology, audits, and rigorous human scepticism.
- [Crypto: Scam or Salvation?](https://ianreppel.org/crypto-scam-or-salvation/) (2025-05-15): cryptocurrencies are neither a panacea nor a Ponzi scheme: they solve niche but real problems like cheap cross-border remittances. Yet less than half a percent of transaction value is retail and remittances, 60% of on-chain volume is driven by bots, and the environmental bill of $17-28 billion dwarfs the benefits. Until blockchains move more real value with less carbon, crypto stays closer to scam than salvation.
- [Bullshit, Lies, or Improvisation?](https://ianreppel.org/bullshit-lies-improvisation/) (2025-05-09): Not all executive waffle is equal: bullshit is indifferent to the truth and aims to impress, lies deliberately conceal it, and improvisation is genuine thinking out loud. Bullshitters speak smoothly with buzzwords, liars leak cognitive load, and improvisers hedge and reason aloud.
- [When Everyone Codes, Who Maintains the Mess?](https://ianreppel.org/when-everyone-codes-who-maintains-the-mess/) (2025-05-06): LLMs make coding easy, but empowerment without accountability merely shifts the burden: when everyone codes, software engineers become glorified janitors paged for messes they never created. Software development is rarely the bottleneck in organizations; coordination and prioritization are. Speed is not the enemy, speed without accountability is.
- [Peso Poor, Dollar Rich: Who Wins Without the Cepo?](https://ianreppel.org/cepo-cambiario/) (2025-05-05): Lifting Argentina's capital controls was hailed as a golden era, but the benefits go to the rich: the 5% who hold the dollars. Prices react to a floating dollar while incomes do not, so life got harder for the 95% with few or no savings. Trickle-down economics is a fallacy, and capital liberalization mostly benefits capital-intensive sectors.
- [The Penalty of Being Human](https://ianreppel.org/the-penalty-of-being-human/) (2025-05-01): Each year, three people die waiting for Cloudflare and Google to verify they are human. CAPTCHAs penalize humans to protect websites, even though AIs are now better at solving them than people. Scaled up, one second of delay for every internet user amounts to 176 years of collective time wasted annually.
- [Argentina's Quantum Leap](https://ianreppel.org/quantum-in-argentina/) (2025-04-24): Argentina can lead Latin America into the quantum age if it has the courage to invest now. Quantum computing is 5-10 years away from mass adoption, with a global economic impact of at least $1 trillion by 2035, yet investment in Latin America is nearly zero. A national quantum strategy, funded research, and quantum literacy in schools would make the workforce quantum native by the time the technology matures.
- [Life in Argentina](https://ianreppel.org/argentina/) (2025-04-24): Argentina's economic reality under Javier Milei can be summed up as: the numbers may be better but life is definitely worse. Inflation has come down from its peak, yet 37% live in poverty, wages stagnate, and deregulation has sent gas, rents, and health insurance soaring. Austerity stabilizes numbers by destabilizing lives.
- [Slow. The Fuck. Down.](https://ianreppel.org/slow-the-fuck-down/) (2025-04-15): The world is sprinting towards a precipice of its own making: sustained growth is physically unsustainable, and speed magnifies mistakes. The tech industry sacrifices collective well-being for engagement and revenue, treating users like addicts. Slow down on purpose, embrace friction, and reclaim your attention before nature does it for you.
- [Engagement: Vanity of Vanities](https://ianreppel.org/engagement/) (2025-04-15): Engagement is the vanity metric of all: it measures addiction, not satisfaction with or utility of a product. A thought experiment shows how optimizing products for engagement conditions unhealthy customer behaviour. Design products for healthy usage, not addiction, and stop treating engagement as a proxy for success.
- [Evacuate the Planet!](https://ianreppel.org/evacuate-the-planet/) (2025-04-08): A back-of-the-envelope calculation shows that evacuating all human beings to Mars is impossible with current and foreseeable technology. The energy required is on the order of the annual global energy supply, and we would need at least 80 million launches. There is no Plan(et) B.
- [Bayes' Rule in Product Management: Sequential Bayes](https://ianreppel.org/bayes-rule-in-product-management-ii/) (2025-03-27): Sequential Bayes models adoption, engagement, and retention as a Markov chain without assuming conditional independence. Each signal boosts the probability of success, from 5% to 40% in total, whereas naive Bayes overestimated it by more than a factor of two. The order matters, and the quality of the joint probabilities is crucial.
- [Bayes' Rule in Product Management](https://ianreppel.org/bayes-rule-in-product-management-i/) (2025-03-27): Bayes' rule lets product managers update the probability of success with fresh data, from sign-ups and negative reviews to A/B test results. With adoption, engagement, and retention as conditionally independent signals, the probability of success rises to almost 90%. Naive Bayes overestimates, though, because the signals are correlated.
- [The Drake Equation for Product Management](https://ianreppel.org/the-drake-equation-for-product-management/) (2025-03-26): The Drake equation can be adapted to product management to compute the number of successful products in the market. Retention and idea generation with validation are the key levers to increase product success.
- [The Death of the Efficient-Market Hypothesis](https://ianreppel.org/the-death-of-the-efficient-market-hypothesis/) (2025-03-24): Jensen Huang admitted he was wrong about quantum computing, yet quantum stocks fell another 10% instead of rebounding. The market did not react to new information; it reacted to the source, an irrelevant one at that. If that is not the final nail in the coffin of the efficient-market hypothesis, nothing is.
- [Leadership Lessons from Jazz, Improv, and Japan](https://ianreppel.org/leadership-lessons/) (2025-03-12): Jazz, improvisation theatre, and Japanese philosophy teach leaders to accept the mess, build on ideas with 'Yes, and...', be present in the moment, and fall down seven times only to stand up eight. Leadership is collective: share the spotlight, hire only the best, and thank the team for their efforts. Playing it safe lets leaders survive, whereas improvising lets them thrive.
- [The Tyranny of the Calendar](https://ianreppel.org/the-tyranny-of-the-calendar/) (2025-03-05): The calendar has become a tyrant we voted to rule over our lives: back-to-back meetings, arbitrary deadlines, quarterly OKRs, and fiscal-year planning that never matches reality. Creativity thrives in unstructured moments, but the calendar leaves no room for them. It is time to take back control and let the calendar serve us, not enslave us.
- [The Myth of the Efficient-Market Hypothesis](https://ianreppel.org/the-myth-of-the-efficient-market-hypothesis/) (2025-03-04): The efficient-market hypothesis ought to be rejected. When Jensen Huang voiced his opinion on quantum computing, quantum stocks tanked despite experts disagreeing, and DeepSeek's unverified claims wiped almost $600b off NVIDIA's market value. The market reacts to irrelevant information and unverified claims, which is not efficiency but stupidity.
- [Product Management: The Cult of Empty Thinking](https://ianreppel.org/the-cult-of-empty-thinking/) (2025-03-03): Product management has become a cult of empty thinking where buzzwords stand in for substance and progress is a PowerPoint slide. Most PMs hoover up requirements, host meetings instead of unblocking engineers, and write vision docs that read like LLM-generated slop. The rare few who ship products that matter are the exception, not the rule.
- [Systems Thinking and Theseus' Ship](https://ianreppel.org/systems-thinking-and-theseus-ship/) (2025-02-27): Systems thinking resolves the Ship of Theseus paradox: a system stays the same when its elements are swapped out gradually because its function and interconnections remain intact. Common philosophical solutions, from Heraclitus to perdurantism, each fail in their own way. The ship, like the human body, is more than the sum of its parts.
- [Quantum Computing Roadmaps in 2024](https://ianreppel.org/quantum-roadmaps-in-2024/) (2024-12-10): Vendor roadmaps for physical and logical qubits, plus qumodes for photonic chips, show that despite impressive progress and media 'breakthroughs', the timeline for quantum advantage remains as originally predicted: from 2030 onwards. A conservative physical-to-logical qubit ratio of 1000:1 suggests 75 to 100 logical qubits are needed for quantum advantage, which is expected near the end of the decade. This applies to big compute, not big data problems.
- [Global Investment in Quantum Computing](https://ianreppel.org/global-investment-in-quantum-computing/) (2024-10-24): As a fraction of GDP, the United Kingdom, South Korea, Singapore, Israel, and Finland invest the most public and private money in quantum technologies, whilst Latin America, the Middle East, and Africa lag severely. Since quantum advantage is likely a decade away, investments now in a quantum-ready workforce, anywhere in the stack, will determine which countries benefit.
- [Icarus and Product Management](https://ianreppel.org/icarus-and-product-management/) (2024-10-01): The myth of Daedalus and Icarus is usually read as a warning against hubris and a means to enforce compliance, but the real moral is for product management and technology innovation. Icarus escaped and died free, not as a prisoner, and he did not display hubris: he was merely mesmerized by the ability to fly, as users are by true innovations. Any product manager knows that if a product can be used in a way for which it was never designed, it will be.
- [The Corporate Village Idiot](https://ianreppel.org/the-corporate-village-idiot/) (2024-09-30): A village idiot is generally harmless until such a person suddenly obtains power, at which point they become a serious problem that undermines morale. Corporate village idiots acquire authority without the competence to wield it, and they come in archetypes: the nuclear shield, the fruit fly, the butterfly chaser, the brilliant dolt, the drinking bird, and the PowerPoint architect. They are distinct from toxic bosses with dark-triad traits, who are deliberately destructive. If you cannot identify the village idiot in an organization, chances are you are the village idiot.
- [Use Cases for Near-Term Quantum Computers](https://ianreppel.org/use-cases-for-near-term-quantum-computers/) (2024-09-06): Across the roughly 70 algorithms on the Quantum Algorithm Zoo, only those with exponential, superpolynomial, or quartic-or-better speed-ups are sufficient for practical quantum advantage on NISQ machines, which covers about half. In quantum machine learning, optimization, and numerics, only 38% of algorithms have even a possible advantage. The best near-term use case remains Feynman's original idea: simulating quantum physical processes in chemistry and materials science.
- [Where Do You See Yourself in Five Years?](https://ianreppel.org/where-do-you-see-yourself-in-five-years/) (2024-02-05): When asked where I see myself in five years, I haven't the foggiest, and that's OK. My path from forensic pathologist dreams to physics research to data engineering to product management was sinuous and not at all planned, which shows there is no single path to becoming a PM and that MBA graduates are not the only ones who can do it.
- [Spinning Quantum](https://ianreppel.org/spinning-quantum/) (2024-02-04): Quantum computing is a furnace for burning venture capital money: it will be transformative once we have useful machines, but we are still very far off from any utility. Quantum supremacy claims proved bogus, the factoring record stands at 21 = 3·7, and the industry is already seeing a quantum winter as VC investment halves and labs shut down. Quantum computing cannot possibly live up to the hype it generates itself. Right now, it is all spin.
- [The Organizational IQ Gradient](https://ianreppel.org/the-organizational-iq-gradient/) (2024-02-03): Hiring people smarter than you at every level produces the organizational IQ gradient: the smartest people end up as individual contributors and the dumbest at the top, on average. The more layers in the hierarchy, the greater the gap, and the simple calculation matches the STEM-to-MBA intelligence gradient seen in tech companies. Generalists versus specialists, emotional intelligence, age-related decline, and the Peter principle all fail to explain it.
- [LLMs vs the Snorbograff: Language Is More Than Statistics](https://ianreppel.org/llms-vs-the-snorbograff/) (2024-01-28): LLMs only know form, not meaning and intent, so they produce bullshit at scale when asked about things they cannot know. Bender and Koller's octopus test exposes this: an octopus that mimics telegraph usage passes the Turing test but cannot help when Amy is attacked by a bear. Testing fifteen LLMs with a prompt about defending against a snorbograff, a creature that does not exist, shows most pretend to know, invent facts, or miss the urgency.
- [The Beach House and the Crapshack](https://ianreppel.org/beach-house-and-crapshack/) (2024-01-21): Tasks sit in the beach house or the crapshack: important tasks you enjoy live in the former, important tasks you do not in the latter. Habituation dulls the beach house, so keep a mental villa in view, and delegate what lands in the crapshack.
- [Reasons for the Collapse of State-Level Societies](https://ianreppel.org/reasons-for-societal-collapse/) (2023-12-31): Analysing the MOROS data set on the mortality of states shows that the average society collapses only 228-326 years after its formation, with nation states faring the worst. Military conquest is the most common cause of collapse, followed by multi-causal systems collapses, while climate change by itself is rarely the cause. Natural causes rarely end civilizations, and rebellion and revolution are rarely found in the same collapse as slow disintegration.
- [PM Interview Questions](https://ianreppel.org/pm-interview-questions/) (2023-12-15): The best interview questions for technical product managers are about reading and thinking, not case studies, because product management attracts bullshitters who can slip through the cracks. Ask candidates to think like a product manager, not a consumer, by dissecting a product live, and to tell you something they learned from a non-fiction book, which reveals whether they can concentrate, think independently, and explain clearly. For TPMs, ask for recent insights from research papers or engineering blogs, because you cannot make people care about staying informed.
- [Maybe Means No](https://ianreppel.org/maybe-means-no/) (2023-12-10): When product managers say 'Maybe' they generally mean 'No', and that is deceit, because it removes the customer's agency to choose a workaround or leave for another product. Saying no to a feature does not close the door on the customer: it erases a sketch of a feature that never existed anyway, and customers rarely switch over a singular absent feature. Be honest without being rude, because a lie is fine only if you are never found out, and corporate dysfunction is not a reason to lie to customers.
- [Placebo Confidence in Deep Tech](https://ianreppel.org/placebo-confidence-in-deep-tech/) (2023-11-23): In deep tech you won't know if it works until you build it, so detailed plans up front are a waste of time and their timelines generate placebo confidence: overall confidence stays stable until the end, when it becomes clear the idea was unattainable. A waterfall/project mindset expects certainty up front and measures success by compliance, while an agile/product mindset accepts that nothing is certain and expects to learn along the way. Be open about risks, state the most important open questions, and say 'I don't know' when you do not, because if people demand certainty where there is none, they can expect to be lied to.
- [The Urgency Test](https://ianreppel.org/the-urgency-test/) (2023-11-22): One question ascertains whether a sudden customer problem is urgent: 'What is the worst thing that could happen if we don't do ... now?' Urgency is not transferable, and the test forces people to think about the worst outcome of not acting immediately, which often reveals the request was not that urgent. If any delay could cause a serious problem, follow up with 'And how likely is such an outcome?' to decide whether the team should refocus.
- [Feature Ideas without Product Debt](https://ianreppel.org/feature-ideas-without-product-debt/) (2023-11-20): A roadmap with four sections, Now, Next, Later, and Maybe, keeps feature ideas visible without accumulating product debt, and the Maybe pile is for internal purposes only. A feature moves forward only if it passes five stages, from a high-level description through a customer value statement and product hypothesis to validation with customer data, after which it is prioritized or dropped. Review items regularly and kill with fire anything that has not moved forward, so the Maybe pile does not become a graveyard of obsolete ideas.
- [Product Debt](https://ianreppel.org/product-debt/) (2023-11-19): Product debt is like technical debt: today's time saver becomes tomorrow's burden, and it tends to turn into technical debt, so developers pay the interest on bad product decisions. It comes from certainty creep, in which uncertainty about a feature morphs into misguided confidence without data or customer research, and it is insidious because people add rather than subtract and anchor on what is available. Add only if you absolutely must, because you rarely subtract once you have committed to a solution, even on the drawing board.
- [AI: Black Magic inside a Black Box](https://ianreppel.org/ai-black-magic-in-a-black-box/) (2023-11-18): Digital products hide black magic inside a black box, and the right to an explanation should not depend on implementation details, yet unregulated AI itself does not increase harm: automation without accountability does. Trust in a system depends on how its decisions affect you, and people accept terrible explanations, so conformity assessments cannot fix the black-box problem. Overly generic legislation that singles out AI is downright suspicious, and without accountability transparency is pointless.
- [500 Days of Logseq](https://ianreppel.org/logseq-500-days/) (2023-11-17): After 500 days, the Logseq knowledge graph has grown larger and more interconnected, mostly as a function of time spent reading and annotating. Readwise's Reader has replaced Feedly, Pocket, and Command, Snipd adds podcast transcripts, and Zotero has replaced Paperpile for research papers. Aliases let you group related topics and tags onto a single page for a better overview of notes.
- [The Captainless Ship of Product](https://ianreppel.org/the-captainless-ship-of-product/) (2023-11-11): A product team is a ship without a captain: the PM is the compass needle, not the commander, and the only power a PM has is influence earned through trust. The real captain is consensus among the triumvirate of product, engineering, and design, who share responsibility without direct authority. But when the ship sinks, the failure falls on the product manager, so with great responsibility comes no power, and if authority is what you seek, product management will suit you ill.
- [Assume Nothing](https://ianreppel.org/assume-nothing/) (2023-11-05): The prime directive of product discovery is to never assume anything: work from first principles, and back any assumptions you do list with customer research and data, revisiting them regularly because unverified assumptions become the gospel. The directive also applies to people: disorder is the natural state, agreement does not magically happen, and a thumbs up in Slack is not consent. Expect that nothing will happen of its own accord, because if you don't make it happen, who will?
- [The QVR Framework for Data Source Valuation](https://ianreppel.org/qvr-framework/) (2023-11-01): The QVR framework answers the which and why (not) questions of data sources, not the what: it cannot find suitable sources, only rank them. Proceed through four stages, discarding sources whose quality is insufficient, whose value does not outweigh costs, or whose risks cannot be mitigated, then compute a QVR score as the product of quality, value, and risk. Quality and value are linked, so the assessment is a combined effort between engineering and product, and a source with no value scores zero regardless of quality.
- [The Dumbest Motherfucker in the Room](https://ianreppel.org/the-dumbest-motherfucker-in-the-room/) (2023-09-10): The hardest lesson for an engineer who turns into a technical product manager is to embrace being the dumbest motherfucker in the room: ask the seemingly obvious questions and act like you know nothing, because you have no idea about your customers' unique environment. Preconceived notions have no place in product discovery, and the true value of a PM is stumbling on an acute but hidden customer pain. Everyone is winging it, so a PM cannot afford to pretend to know more than they do, and being uncomfortable is the job.
- [Introducing: Quantum Insights](https://ianreppel.org/launching-quantum-insights/) (2023-08-30): Quantum Insights is now available on ianreppel.org: a section that discusses interesting research in quantum computing and quantum information science, placing each paper in a larger context, with a fresh insight every few weeks.
- [Why Do We Automate the Arts Away with AI?](https://ianreppel.org/ai-in-the-arts/) (2023-08-20): We automate away the arts, the few things we truly enjoy, instead of the menial chores that dominate our lives, because chores are unpaid work and profiteering off artists enriches the few. Exploitation is not innovation: studios and labels want to scan actors and copy voices without consent, and the end game is AI-generated content optimized for engagement. STEM can increase our life span, but it is the humanities that make life worth living, and just because we can automate the arts does not mean we should.
- [The Quantum Stack](https://ianreppel.org/the-quantum-stack/) (2023-08-15): A quantum computer is a specialized accelerator, like a GPU, best for algorithms that leverage quantum parallelism, and quantum computers of the foreseeable future will be mostly useful for quantum simulations because of the I/O bottleneck. You program a quantum algorithm in a language compiled into the chip's instruction set, with Python likely to emerge as the primary language. Until fault-tolerant machines arrive in a decade or so, algorithm designers must handle error correction themselves, and surface-code redundancy can require more than 1,000 physical qubits per logical qubit.
- [Making Quantum Computers](https://ianreppel.org/making-quantum-computers/) (2023-08-15): Superposition, quantum parallelism, and entanglement are the three key ingredients for qubits, and the DiVincenzo criteria are the cooking instructions for turning them into practical quantum chips. Quantum parallelism is baked into the machine by quantum physics, but the speed-up is exponential only for a few algorithms such as Shor's, HHL, and QFT. Entanglement is a necessary but not sufficient condition for quantum advantage, and the many qubit modalities differ in gate fidelity, coherence times, and gate speed.
- [High-Temperature Superconductors in Quantum Computers](https://ianreppel.org/high-temperature-superconductors-in-quantum-computers/) (2023-08-01): Even if LK-99 turns out to be a veritable room-temperature superconductor, it would not help superconducting quantum computers with current designs and electronics. Qubits must be cooled to minimise thermal noise, and tweaking the energy levels lands you in the terahertz gap where no commercial electronics exist to manipulate the qubits. Decoherence, not induced relaxation, is the more worrisome problem, and it still demands extremely low temperatures.
- [The Quantum Collection](https://ianreppel.org/quantum-guide/) (2023-07-25): A curated guide to the quantum computing posts on this site, organized by reader intent: explorers, learners, those seeking inspiration from real use cases, researchers, and readers who want the hype managed. The most viable near-term use case for NISQ devices is the simulation of quantum physical processes, not optimization, machine learning, or cryptography.
- [The Failure of the GDPR](https://ianreppel.org/gdpr-failure/) (2023-07-24): Five years on, the GDPR's primary achievement is €4b in fines, yet personal data of Europeans is still shared 376 times each day for online advertising and the same companies pop up time and again. Large tech firms treat penalties as the cost of doing business, and consumers appear not to care enough to switch, so trust is not correlated with market success. The AI Act reuses the GDPR playbook of vague definitions and self-regulation, so it may achieve the opposite of its intentions.
- [The I/O Bottleneck in Quantum Computing](https://ianreppel.org/io-bottleneck-in-quantum-computing/) (2023-07-22): Quantum computers of the next few years will be suitable for simulating quantum physics in chemistry and materials science, not for problems that require large amounts of data. The I/O bottleneck, not classical simulation limits, is the constraint: a 100-logical-qubit machine has a peak bandwidth two orders of magnitude below a modern CPU, and no realistic QRAM exists. Quantum computers will be practical for 'big compute' problems on small data, not big data problems.
- [A Review of the Best Quantum Computing Courses for All Skill Levels](https://ianreppel.org/a-review-of-quantum-computing-courses/) (2023-06-01): MIT's Quantum Computing Fundamentals together with IBM's free Qiskit Summer School offer the best foundation and practice for serious learners. MIT's Quantum Computing Realities is technically excellent but heavy on density matrices and error correction, while Oxford's two-day course is short and sweet but hard to swallow at the price. Purdue, TU Delft, CERN, AWS, and LinkedIn offerings disappoint or are too niche.
- [Running Meetings](https://ianreppel.org/running-meetings/) (2023-02-26): No agenda, no meeting: decline invitations without one. Document decisions, assign owners, and share notes so everyone can object or catch up. Start on time, invite only those needed (ten at most, three to five for technical topics), and use a strategic pause, round-robin solicitation, a timeout on talkers, and raised hands to keep the meeting productive.
- [Useful Research Tools](https://ianreppel.org/research-tools/) (2023-02-25): Perplexity is an answer generator powered by a search engine, preferable to the half-baked AI search from Microsoft and Google. Connected Papers and Inciteful visualise citation graphs to surface overlooked literature, Shortform's book summaries are exceptional and worth the price, and Kagi's custom bangs hand queries off to other tools without switching search engines.
- [Tips for Presentations](https://ianreppel.org/tips-for-presentations/) (2023-02-19): Don't write every word of what you are going to say: slides support the talk, so put only a one-liner, quotation, picture, or 3-5 headline bullet points on each. Spend at least a minute per slide, skip special effects, and use colour wisely, keeping in mind that red/green colour blindness affects 10% of people.
- [Thoughts on Writing](https://ianreppel.org/thoughts-on-writing/) (2023-02-18): Write the way you code: plan for five versions, from a hacky first draft to a polished piece for wider readership. Lead with the bottom line (BLUF) so readers get your main point within 10 seconds, and write for the reader, not yourself. Clarity begins and ends with the author, so sleep on it, kill sentences you love if they ruin the flow, and make one crucial point a few times in different guises.
- [PM, Pope of Nope](https://ianreppel.org/pope-of-nope/) (2023-02-08): A good product manager is the Pope of Nope: saying no by default and only saying yes when reasonably confident a solution is valuable, viable, and feasible. Products and features are worthless until proven valuable, and the burden of proof falls on the PM. 'I'll put it in the backlog' is just PM-speak for no, and zombie requests come back to haunt every prioritization session.
- [Research Papers with Logseq](https://ianreppel.org/research-papers-with-logseq/) (2023-01-30): Logseq handles research paper annotations well enough that I switched from Paperpile for that task.
- [The Product Management Opportunity in Quantum Computing](https://ianreppel.org/the-product-management-opportunity-in-quantum-computing/) (2022-12-06): Surveys show vendors and early adopters disagree on the top priorities for near-term quantum use cases: vendors emphasize optimization while industry practitioners prefer quantum machine learning. Without modern product management practices, quantum vendors risk falling into the build trap and achieving poor product/market fit across hardware, software, and services.
- [15 Questions about Quantum](https://ianreppel.org/questions-about-quantum/) (2022-12-05): Fifteen frequently asked questions about quantum computing, covering what qubits are, why quantum advantage is expected between 2030 and 2040, how quantum computers differ from classical ones (superposition, entanglement, no cloning), and what near-term use cases exist. Quantum computers are co-processors, not replacements for classical machines, and more than a quarter of Fortune Global 500 companies are already exploring them.
- [DevOps for ML](https://ianreppel.org/ml-factory/) (2022-11-27): In ML, we ship the factory, not merely the final product: the entire pipeline that regenerates data, features, and models must be deployed, not just the trained model artefact. A feature store decouples DataOps from MLOps, so MLOps ships the assembly line while D/MLOps ships the whole factory.
- [Quantum Tech in the Global 500](https://ianreppel.org/quantum-tech-global-500/) (2022-11-20): Europe leads Fortune Global 500 quantum adoption at 30%, with Japan (22), the US (19), and Germany (13) as the top countries. The top use cases are optimization, material simulations, and machine learning; IBM is the preferred partner overall, D-Wave leads in optimization, and JPMorgan Chase tops the list by number of quantum use cases.
- [Near-Term Quantum Use Cases](https://ianreppel.org/nisq-use-cases/) (2022-11-19): In the NISQ era (up to roughly 2030), the most promising quantum use cases are simulations in materials science and chemistry, which can achieve exponential speed-ups without huge data volumes. Quadratic speed-ups alone cannot overcome the I/O bottleneck between classical and quantum computers, so common applications like machine learning and database searches are not expected to deliver quantum advantage soon.
- [Zoose on GitHub Codespaces](https://ianreppel.org/zoose-codespace/) (2022-11-14): Zoose on GitHub Codespaces offers an installation-free developer experience: two clicks spin up VSCode in the browser with Qiskit, Braket, Cirq, Pennylane, pytket, QuTiP, or PyTorch pre-installed. An alternative to Zoose for Gitpod.
- [Run PyTorch in Your Browser](https://ianreppel.org/pytorch-with-zoose-for-gitpod/) (2022-11-03): Running PyTorch usually means choosing between a browser notebook and a local IDE, with setup overhead either way. Zoose for Gitpod gives you both (VSCode and JupyterLab) in the browser with zero installation: fork the template, open a Gitpod workspace, and PyTorch 1.13 with Audio, Text, Vision, and TensorBoard is ready. PyTorch covers roughly 60% of all papers with code and models on Hugging Face, making it the default for replicating studies and integrating state-of-the-art models.
- [Zoose 3.0 ♥ Gitpod](https://ianreppel.org/zoose-3.0/) (2022-10-15): Zoose for Gitpod offers a browser-based Python notebook experience with VSCode and JupyterLab, requiring no local installation of Python or Docker. Ephemeral workspaces are exact copies of one another, making it easy to replicate development environments in a team.
- [Roads to Quantum Advantage](https://ianreppel.org/roads-to-quantum-advantage/) (2022-09-08): Quantum advantage, when quantum computers outperform classical ones at realistic problems, is expected between 2030 and 2040 at around 70-75 logical qubits. Comparing physical and logical qubit roadmaps across manufacturers shows most expect to reach this threshold by 2030-2035, with PsiQuantum and Google offering more aggressive timelines.
- [Learn Quantum Computing](https://ianreppel.org/learn-quantum-computing/) (2022-09-07): A curated list of articles, books, videos, courses, and interactive programmes for learning quantum computing, spanning beginner introductions to advanced topics like quantum error correction and quantum machine learning. Recommendations include the Qiskit textbook, IBM Qiskit Summer School, TU Delft's Quantum 101, and the quantum algorithm zoo.
- [Aegean Bronze Age Chronology](https://ianreppel.org/aegean-bronze-age-chronology/) (2022-09-05): An interactive timeline for the Aegean Bronze Age based on the high chronology, covering the Cyclades, Minoan Crete, Mycenaean Greece, Troy, and Ancient Egypt. Modern scientific methods such as carbon dating and thermoluminescence all favour a high chronology over the low chronology commonly used in sources.
- [What is Quantum Computing?](https://ianreppel.org/what-is-quantum-computing/) (2022-08-29): Quantum bits can be both zero and one simultaneously (superposition), and entangled qubits create correlations that enable exponential speed-ups for certain algorithms. Quantum computers will serve as co-processors (QPUs) rather than replace classical computers, with applications in logistics, materials science, chemistry, and drug discovery across seven hardware modalities: superconductors, ion traps, photonics, topological superconductors, quantum dots, neutral atoms, and quantum annealers.
- [Introducing Zoose 2.0](https://ianreppel.org/zoose-2.0/) (2022-08-22): Zoose 2.0 splits into separate Docker images: a base Python image, an R flavour, a Neo4j flavour, and a new Zoose Quantum image with Braket, Cirq, OpenFermion, Pennylane, pytket, Qiskit, QuTiP, and Strawberry Fields pre-installed. No setup of virtual environments is required, just Docker.
- [PKM with Logseq and Readwise](https://ianreppel.org/knowledge-management/) (2022-08-01): After trying OneNote, Evernote, Obsidian, NotePlan, and others, I settled on Logseq connected to Readwise for personal knowledge management. Readwise automatically synchronises highlights from Command, Feedly, Kindle, and Pocket into Logseq, where basic and advanced queries can surface combinations of tags across all notes.
- [A Brief History of Quantum Programming](https://ianreppel.org/a-brief-history-of-quantum-programming/) (2022-07-29): A timeline of quantum programming software from QCL (2000) through 2023, tracking SDKs, simulators, assemblers, and cloud platforms. Most publicly visible activity in quantum software has happened in the last five years, with bold entries marking where code can run against actual quantum hardware.
- [Introducing Zoose](https://ianreppel.org/zoose/) (2022-03-01): Zoose is an open-source Docker container image for Jupyter notebooks, pre-loaded with Python and R packages for data science and a Neo4j web server for graph analytics. Containerized notebooks eliminate virtual environments and brittle configurations, and versioned images make sharing and reproducing notebooks straightforward.
- [Learn Audio Engineering](https://ianreppel.org/audio-engineering/) (2021-09-06): Three online audio engineering courses reviewed: Audio Institute of America teaches theory but lacks hands-on exercises, Alan Parsons' ASSR shows real studio work but offers no structured feedback, and HOFA College provides professional mix reviews with detailed critique. HOFA is the best by a landslide: there is no substitute for hands- and ears-on training with regular feedback from the pros.
- [The 70 + 3×10 Model](https://ianreppel.org/70-plus-3x10-model/) (2021-08-02): The 70+3x10 model splits engineering time into 70% development, 10% maintenance and support, 10% operations, and 10% slack for learning. Slack can temporarily absorb spikes in maintenance or operations, but sustained increases must be prioritized in the development roadmap.
- [Product Prioritization with ICE·T](https://ianreppel.org/icet-prioritization/) (2021-07-27): ICE·T adds time criticality to the ICE scoring model, yielding four factors: impact, confidence, ease, and time, each on simple three-level scales. The product of these factors produces 31 unique scores. ICE·T applies to new and existing products, both external and internal.
- [ML Cards for D/MLOps Governance](https://ianreppel.org/ml-cards/) (2021-05-11): ML cards are standardized templates for documenting data, features, models, services, and metrics across D/MLOps. Each card type is combined with basic cards for code, cost, and lineage to provide end-to-end governance.
- [Sorry, Batman is Busy](https://ianreppel.org/how-to-succeed-in-ml/) (2021-05-10): Machine learning failure rates sit at 85% to 87%, and six traps explain why: governance, operationalization as an afterthought, technology as a universal solution, hiring practices, attitudes, and management. The fix is to think in terms of ML products instead of one-off projects, and to deploy the factory, not just the final product. Cross-functional product teams with D/MLOps practices are the foundation.
- [Dimensions of Data Quality](https://ianreppel.org/data-quality-dimensions/) (2021-05-05): Data quality must be assessed along eight dimensions: availability, timeliness, freshness, completeness, validity, consistency, correctness, and trustworthiness. Unless data stewards have both authority and technology to expose and fix issues, data will always be messy.
- [Personal Priorities](https://ianreppel.org/personal-priorities/) (2021-05-03): People who buy into hustle culture struggle to find meaning outside of work, because hobbies must be monetized and turned into side hustles. Two thought experiments help: imagine your death bed to see what truly matters, then picture your retirement speech to factor in the reality of earning a living. The most valuable aspect is learning when to say 'No' and why.
- [So You Want to Be a Billionaire?](https://ianreppel.org/billionaires/) (2021-05-01): The Forbes 400 claims 70% of billionaires are self-made, but only 35% came from lower- or middle-class backgrounds, and the actual odds for someone in that bracket are roughly 1 in 2.3 million. The majority are products of privilege and dumb luck. Becoming a billionaire is an asinine career goal: the world does not become a better place for the rest of us with yet another money hoarder.
- [Of Research and Rogues](https://ianreppel.org/research-and-rogues/) (2021-04-30): Media coverage of machine learning focuses almost exclusively on research breakthroughs, misleading novice data scientists and business leaders into thinking ML is about modelling instead of operations. Reproducibility, a core tenet of the scientific method, has been largely abandoned in ML research, with only 15% of studies publishing their code. Rigour, reproducibility, requirements specifications, and robustness must be central to industrial ML.
- [How to Spot a Rogue Data Scientist](https://ianreppel.org/how-to-spot-a-rogue-data-scientist/) (2021-04-30): A rogue data scientist is anyone in industry paid to deliver results through ML who has little incentive or interest to operationalize their work, working mostly for their own intellectual enrichment. The telltale signs split into academic mindset and résumé padding. A single rogue can contaminate an entire data science organization when left unchecked by management.
- [DevSecOps for ML](https://ianreppel.org/devsecops-for-ml/) (2021-04-21): Distroless container base images cut ML container size by 52% and reduce vulnerabilities by 99%, from 275 to 3. A multi-stage Docker build copies Python site-packages into a distroless image, eliminating the OS layer and its CVEs.
- [Machine Learning Industry Research](https://ianreppel.org/ml-industry-research/) (2021-02-08): A curated aggregation of industry research on enterprise machine learning, organized as highlights and detailed insights with source links and publication dates so you can quickly decide whether each insight is relevant and current enough.
- [Machine Learning Platforms in 2021](https://ianreppel.org/ml-platforms-in-2021/) (2021-02-02): Kubernetes has become the standard infrastructure for machine learning platforms, powering more than 60% of non-managed end-to-end platforms. Kubeflow Pipelines is used by 38% of Kubernetes-based platforms. Feature stores remain rare in commercial offerings, and air-gapped deployment support is uncommon despite demand from manufacturing, government, and finance.
- [What is a Feature Store?](https://ianreppel.org/what-is-a-feature-store/) (2021-01-27): A feature store is a data management layer that saves features specifically designed for machine learning use cases. Its time travel capabilities enable point-in-time backfills for compliance, debugging training/serving skew, and fixing data leakage.
- [A Brief History of Machine Learning Platforms](https://ianreppel.org/a-brief-history-of-ml-platforms/) (2020-09-23): A timeline of data processing, ML, and orchestration technologies from OpenCV in 2000 through to 2022, tracking when end-to-end ML platforms emerged at major tech companies. Facebook's FBLearner Flow in 2016 was the first to be publicly detailed; by 2020 more than a dozen companies had announced theirs. The space has not yet converged on a dominant design.
- [Open-Source Data and Machine Learning Software by the Numbers](https://ianreppel.org/open-source-data-and-machine-learning-software-by-the-numbers/) (2020-03-16): GitHub stars and Google Trends data reveal where the industry converges: Apache Spark leads data processing, Airflow dominates workflow management, TensorFlow rules ML frameworks, and Kubernetes owns container orchestration. GitHub community activity does not always align with real-world adoption, as DB-Engines rankings for databases show.
- [Business Email Etiquette](https://ianreppel.org/business-email-etiquette/) (2020-03-13): Subject line prefixes like [ACTION], [DECISION], [INFO], and [REVIEW] help recipients triage emails quickly. Lead with the request (BLUF), limit CC'd recipients to those who genuinely need the thread, and include a clear deadline in the body.
- [How to Boost Your GCP Productivity in Chrome](https://ianreppel.org/how-to-boost-your-gcp-productivity-in-chrome/) (2020-03-09): aText URL aliases and the Vimium Chrome extension let you reach any GCP console page, API doc, or common service from the keyboard in a few keystrokes. The post includes a table of aliases for GCP, APIs, and general productivity URLs.
- [Shell Scripts for Virtual Python Environments](https://ianreppel.org/shell-scripts-for-virtual-python-environments/) (2020-03-06): A set of shell functions that manage Python virtual environments from the command line: create (pyenv), activate (py), list (pyenvs), delete (nopyenv), and install dependencies from requirements.txt (pipins). All environments live under ~/Python and the create function automatically installs Black.
- [Scalafmt with Docker](https://ianreppel.org/scalafmt-with-docker/) (2020-03-04): Scalafmt formatting results differ across versions even with the same configuration file, because the version field is ignored. Bundling a specific Scalafmt version in a Docker image guarantees consistent formatting across the team.
- [Essential Reading for Data and Machine Learning](https://ianreppel.org/essential-reading-for-data/) (2020-02-25): A reading list for the data and machine learning space: foundational Google papers, core texts on data systems and software architecture, classic ML textbooks, and leadership books. The bias section, anchored by Weapons of Math Destruction, Invisible Women, and Technically Wrong, is the part most people in data tech skip and shouldn't.
- [A Tour of End-to-End Machine Learning Platforms](https://ianreppel.org/a-tour-of-end-to-end-ml-platforms/) (2020-02-21): A survey of end-to-end ML platforms from Google (TFX), Uber (Michelangelo), Airbnb (Bighead), Netflix (Metaflow), and Lyft (Flyte), alongside cloud offerings from AWS, GCP, and Azure, and open-source options like Kubeflow and MLflow. As little as 5% of production ML code is the model itself; the rest is data management, deployment, and monitoring. No single platform has converged as the dominant design.
- [Take-Home Interview Challenges are Biased](https://ianreppel.org/take-home-interview-challenges-are-biased/) (2020-02-14): Take-home interview challenges assume candidates have unlimited free time, a dedicated development machine, and broadband access. Those assumptions systematically disadvantage women, who do the majority of unpaid domestic work, and people from lower socioeconomic backgrounds. The result is a hiring filter that selects for the stereotypical young, white, male 24/7 coder.
- [The Quit Upwards Paradox](https://ianreppel.org/the-quit-upwards-paradox/) (2020-02-13): Employees who quit upwards secure roughly 15% pay rises, yet top performers who stay max out at 5%, creating a paradox of why companies hire externally rather than promote internally. Risk aversion, perception, professional hysteresis, and the reverse Groucho Marx effect all play a role, but none resolves it alone.
- [Lean Data and Machine Learning Operations](https://ianreppel.org/lean-dml-operations/) (2019-12-13): Lean Data and Machine Learning Operations applies manufacturing's continuous improvement philosophy to D/ML pipelines through the systematic elimination of waste. Waste manifests as data transfer, friction from missing operations manuals, storage of unused code and data, built-in scheduler delays, data deluge, extra features, unused skills, and bugs. Kaizen, genba, andon, the 5S methodology, and kanban-style scheduling all transfer directly from the factory floor to operations.
- [Migrations: from Wordpress to Firebase](https://ianreppel.org/migrations/) (2019-12-11): The blog migrated from Wordpress.com (for ease) to Bitbucket-served Jekyll (for Markdown, git versioning, and no ads) to Firebase Hosting (for custom domain ianreppel.org, SSL, and free static hosting). Each move stripped away friction and gave more control over content and delivery.
- [How to Get Started with Scala](https://ianreppel.org/how-to-get-started-with-scala/) (2019-12-02): A curated guide to learning Scala, from general references like Scala for the Impatient and Twitter's Scala School through functional programming resources (Alvin Alexander's Functional Programming Simplified, the red book) to testing with ScalaTest and ScalaCheck, and sbt for build management. Scala's advantage is a type-safe, functional layer on the JVM with a rich ecosystem and REPL for fast experimentation.
- [2BIG in DevOps](https://ianreppel.org/2-big-in-devops/) (2019-11-25): 2BIG is a principle for DevOps: Better Is Good (Occam's Razor for development, ship what is fit for purpose) and Boring Is Good (predictable technology eliminates toil and reduces operational risk). The price of reliability is the pursuit of the utmost simplicity. Technology migrations are fine when there are pressing business reasons, but the desire to experiment with the latest fad must be weighed against the need for reliable software.
- [How to Stay Up to Date with Trends in Tech (Revisited)](https://ianreppel.org/how-to-stay-up-to-date-with-trends-in-tech-revisited/) (2019-11-22): RSS feeds through Feedly, targeted newsletters, the Researcher app for filtered academic papers, and Paperpile for organizing references form a system for keeping up with developments in data, machine learning, and tech leadership without drowning in noise. The Morning Paper is singled out as the single best investment for broadening technical horizons. Quality sources are few; frequent-posting announcement boards are best avoided.
- [Optimization with TensorFlow](https://ianreppel.org/optimization-with-tensorflow/) (2019-08-28): TensorFlow's optimizers, from basic GradientDescent to Adam and FTRL, can solve numerical optimization problems using automatic differentiation, and the post demonstrates this with concrete loss functions and Docker-based setup. Adam is recommended as the default optimizer, with TensorBoard providing visualization of convergence. TFX provides the end-to-end pipeline infrastructure from data ingestion through model serving.
- [Principles of Product Management](https://ianreppel.org/principles-of-product-management/) (2019-08-27): A distillation of product management principles from observation and experience: the PM as compass needle (not the compass), the primacy of impact over action over talk, the team shield role, root-cause questioning through 5x 'Why?', and the discipline of documented decisions. Santa Claus tasks and crystal balls have no place on a task board. Meetings without agendas are to be declined automatically.
- [Joins in Scio](https://ianreppel.org/joins-in-scio/) (2019-08-15): Scio supports inner joins, outer joins, hash joins using side inputs for small right-hand datasets, co-group operations, and MultiJoin for joining up to 22 collections. Hash joins replicate the RHS as a Map side input to avoid shuffling the LHS. Skewed joins fragment the RHS into hot and cold sets, and sparse joins use bloom filters when the RHS is too large for memory.
- [A Beginner's Guide to Stereoscopic Photography on a Smartphone: Kúla Bebe](https://ianreppel.org/a-beginners-guide-to-stereoscopic-photography-on-a-smartphone-part-3/) (2019-07-23): The Kula Bebe is an inexpensive mirror attachment that lets smartphone cameras capture simultaneous stereo pairs, enabling 3D photography of scenes in motion that sequential shooting cannot handle. The fixed 4.5 cm stereo base limits closest objects to 135 cm, and the companion Kulascope app manages edge cropping and haziness from stray light. The best camera is the one in your pocket.
- [Differences in Conversions of Java Numbers in Scala 2.11, 2.12, and 2.13](https://ianreppel.org/differences-in-conversions-of-java-numbers-in-scala/) (2019-03-21): Scala 2.12 and 2.13 silently convert null Java boxed primitives to default values instead of throwing NullPointerExceptions like 2.11 did, which breaks the safeOption pattern that worked in earlier versions. The fix requires an additional implicit conversion parameter and an explicit null check before wrapping in Option. A try-catch approach consistently outperforms Try monad alternatives in benchmarks.
- [Associativity in Semigroups in Scala: Algebird, Cats, and Scalaz](https://ianreppel.org/associativity-in-semigroups-in-scala/) (2018-11-09): Algebird, Cats, and Scalaz do not perform compile- or run-time checks on the laws governing their algebraic type classes, so you can create a class that is an instance of a semigroup while violating associativity. Closure is trivially satisfied by Scala's type system, but associativity is not, and a property-based test with ScalaCheck (using forAllNoShrink to avoid misleading shrinking) is the way to catch it. Cats offers built-in law checks as wrappers around ScalaCheck; Algebird and Scalaz do not.
- [A Beginner's Guide to Stereoscopic Photography on a Smartphone: Pitfalls](https://ianreppel.org/a-beginners-guide-to-stereoscopic-photography-on-a-smartphone-part-2/) (2018-10-09): No parallax means no depth. Beginners make five common mistakes: scenes without foreground depth look flat, baselines too small or too large destroy perspective, aligning the foreground instead of the focal point flattens the subject, motion between sequential shots creates ghosting, and yaw adjustments from a wrong baseline ruin close-ups. Use the 1:30 rule (baseline equals one thirtieth of the distance to the nearest object) and ensure the scene has discernible perspective.
- [A Beginner's Guide to Stereoscopic Photography on a Smartphone: Basics](https://ianreppel.org/a-beginners-guide-to-stereoscopic-photography-on-a-smartphone-part-1/) (2018-10-09): Stereoscopy is the principle behind human vision: two eyes a distance apart see the world slightly differently and the brain fuses the images into depth. You can recreate that effect with any smartphone by taking two sequential pictures from slightly different angles, no special stereo camera needed. Camera 3D Pro and i3DSteroid are the best beginner apps, and a basic setup is enough to make decent stereoscopic photographs.
- [Passing Functions in Scala](https://ianreppel.org/passing-functions-in-scala/) (2018-06-01): Scala offers three ways to pass a function to another: by value, by name, and as a function, and the differences are subtle enough to breed surprising bugs. Passing by value evaluates the argument once before the body runs, so a UUID generator yields one constant value; passing by name or as a function re-evaluates on each use. A def without a parameter list is semantically equivalent to a value thanks to referential transparency, so be careful when you intend to evaluate a passed function multiple times.
- [Weekend Reading Round-Up (Dec. '17)](https://ianreppel.org/weekend-reading-roundup-december-2017/) (2017-12-21): A round-up on machine learning, critical transitions, and music: auto-tuned models (ATM) automate model selection and hyperparameter tuning, and a study of 8,200 world-music recordings flagged 20% as outliers. Hidden early-warning signals in scale-free networks require the effective state rather than the natural state, Deep Rewiring trains very sparse networks, and JamBot composes polyphonic music with chord progressions guiding the melody.
- [Weekend Reading Round-Up (Nov. '17)](https://ianreppel.org/weekend-reading-roundup-november-2017/) (2017-11-17): A round-up on deep learning, leadership, and music: DeepXplore uses differential testing to find corner cases that reveal bugs in neural networks in seconds, and a single-pixel attack can fool image classifiers. Leader gender made no significant difference to team cohesion, and musical genres can be identified from entropy alone by analysing MIDI files.
- [Weekend Reading Round-Up (Oct. '17)](https://ianreppel.org/weekend-reading-roundup-october-2017/) (2017-10-13): A round-up of research across data, machine learning, and mathematics: topological data analysis builds simplicial complexes to infer structure, and teaching personal initiative beats traditional business training, lifting micro-enterprise profits 30% in Togo. Deep reinforcement learning results are hard to reproduce because hyperparameters and implementation details dominate, and TFX shows that notebooks are terrible for deploying models to production.
- [Weekend Reading Round-Up (Sept. '17)](https://ianreppel.org/weekend-reading-roundup-september-2017/) (2017-09-15): A round-up of recent research: a random-forest classifier predicts a graph's category with 94.2% accuracy, reversible residual networks match or beat existing methods with less memory, and phantom types in Scala enforce the builder pattern at compile time. A P300 brain-computer interface lets people compose music by thought, happy music improves divergent thinking, and struc2vec learns node representations from structural identity.
- [Blockchains and Music](https://ianreppel.org/blockchains-and-music/) (2017-03-03): There is no central system tracking who performed a song and who owns its rights, and the consequences are real: Spotify settled a $30m lawsuit because it did not know whom to pay, and 107% of the rights to one song were once sold. A blockchain offers a distributed, immutable, trustless ledger that could register rights and, via smart contracts, make fast, frictionless royalty payments, but the labels' main obstacle is transparency, since everyone wants a light and airy apartment and few want to live in a glass box.
- [Reading JSON Resource Files in Apache Spark](https://ianreppel.org/reading-json-resource-files-in-apache-spark/) (2017-02-17): To read a JSON file from src/main/resources in Spark 2.1 with Scala 2.11, read the stream, trim and collapse whitespace, then parallelize it into an RDD or hand it to sparkSession.read.json for a Dataset. Do not specify a schema for anything other than String unless you are sure every key-value pair is consistent, because the parser's limited type coercion can silently turn an entire row to NULL on a single failed cast.
- [How Do Musicians Make Money? (Part 2/2)](https://ianreppel.org/how-do-musicians-make-money-part-2/) (2017-02-12): Streaming pays artists fractions of a cent per stream, so all but the most popular make very little from it, and the flow of money from fans to artists is a tangled mess of royalties split between songwriters and performers.
- [How Do Musicians Make Money? (Part 1/2)](https://ianreppel.org/how-do-musicians-make-money-part-1/) (2017-02-12): For most of history, professional musicians were slaves, servants, or in direct employment of the nobility. From the Renaissance onwards three models for performers survive to this day: patronage, performance (touring), and residence (tourism). The playback era added publication and royalties, but radio and MTV repeatedly traded exposure for payment, a bargain that works for superstars and not for aspiring talent.
- [Shell Scripts to Check Data Integrity in Hive](https://ianreppel.org/shell-scripts-to-check-data-integrity-in-hive/) (2017-02-11): Hive has no out-of-the-box way to check for duplicate rows or inspect column contents, so a shell script fills the gap. countHiveDuplicates uses ROW_NUMBER() over a PARTITION BY of the chosen columns to count duplicates, and analyseHiveTable builds a LATERAL VIEW explode query that reports min, max, max length, distinct percentage, and null percentage per column, treating everything as strings so it works even on CSV-backed external tables.
- [Shell Scripts to Ease Life with Hadoop](https://ianreppel.org/shell-scripts-to-ease-life-with-hadoop/) (2017-02-10): Interacting with HDFS and YARN via their shell commands is cumbersome, so a library of Bash functions makes it bearable. hconf reveals cluster defaults in one call, hdp and friends abbreviate hdfs dfs, and __hdfs_folder normalizes paths relative to HDP_DATA; hrm even refuses to remove the default data location as a sanity check. ylist, ylog, and yerr package the YARN commands you type most.
- [Collecting Table and Column Statistics in MemSQL](https://ianreppel.org/collecting-table-and-column-statistics-in-memsql/) (2017-02-05): MemSQL does not automatically collect table and column statistics, which the query optimizer depends on, so flushing, optimizing, and analysing ought to be automated. A lightweight shell script driven by two config files runs OPTIMIZE TABLE FLUSH, OPTIMIZE TABLE, and ANALYZE TABLE in parallel across tables, then collects range statistics on specified columns in sequence to avoid hammering the database.
- [How to Stay Up to Date with Trends in Tech](https://ianreppel.org/how-to-stay-up-to-date-with-trends-in-tech/) (2017-01-23): Keeping abreast of tech comes down to a few tools: Hacker News for the quality of its comment discussions, RSS feeds (RSS is not dead) for the bulk of reading, Reddit subreddits, and podcasts. Tech books have such a short shelf life that owning them physically is not worth it, so a Safari Books subscription is the better buy.
- [Playing with Spark in sbt](https://ianreppel.org/playing-with-spark-in-sbt/) (2017-01-13): The standard Spark VMs and Docker images do not offer the latest and greatest, so to get the bleeding edge you can run Spark straight from sbt with a minimal build.sbt and the sbt console. A simple SparkConf and SparkSession setup gives you a shell-like environment to tinker with Spark in the command line or your IDE, no cluster required.
- [A Glossary of Some Software Terminology](https://ianreppel.org/a-glossary-of-some-software-terminology/) (2016-12-08): Software terminology from version control to serverless, each entry structured as what it is, why it matters, the tools, prerequisites, and buzzwords. The glossary is deliberately incomplete, but it draws the distinctions that matter: continuous delivery and continuous deployment are often conflated, containers share the host OS while VMs carry their own, and microservices differ from SOA in being independently deployable. Serverless is an accepted misnomer for ephemeral hosted services.
- [The Problems with Visual Programming Languages in Data Engineering](https://ianreppel.org/the-problems-with-visual-programming-languages-in-data-engineering/) (2016-09-22): Most objections to visual programming languages in data engineering are matters of background, experience, and preference, but three problems are intrinsic to the tools: the near-impossibility of integration into non-monolithic enterprise architectures, a natural resistance to data governance because anyone can be a data engineer, and the promotion of unhealthy habits from extended mouse usage. A decision in favour of a visual data engineering tool is an implicit repudiation of data governance. The claim that data flows are self-documenting is nonsense, and an ETL flow with 200 blocks is about as clear as mud.
- [Batch-Updating Non-Key Columns in Apache Phoenix](https://ianreppel.org/batch-updating-non-key-columns-in-apache-phoenix/) (2016-07-15): Phoenix offers no native `UPDATE` syntax, only `UPSERT`, and because primary keys are HBase row keys there is no way to update key columns at all, only delete-and-insert. To batch-update non-key columns you must `UPSERT` a `SELECT` that re-supplies the key columns alongside the new expression, filtered by a predicate. Salt buckets must be set at table creation and ideally match the number of region servers.
- [Shell Scripts to Ease Spark Application Development](https://ianreppel.org/shell-scripts-to-ease-spark-application-development/) (2016-04-25): Spark application skeletons are all the same except for the main entry point, so a couple of shell scripts automate the boilerplate. `sparkSetup.sh` generates an sbt-based Spark/Scala project scaffold with ScalaTest and sbt-assembly included, while `updateAppConfig.sh` reads your system's Scala and sbt versions and checks Maven for newer versions of ScalaTest, Spark, and Hadoop to keep `app.cfg` current.
- [Spark Actions, Laziness, and Caching](https://ianreppel.org/spark-actions-laziness-and-caching/) (2016-04-01): Transformations on Spark RDDs are lazy and only evaluated when an action forces Spark to come up with an answer; before you call an action nothing gets computed, only the DAG is built. `cache()` and `persist()` are also evaluated lazily and their computations are triggered by actions, so get the timing right when persisting and unpersisting.
- [An Overview of Apache Streaming Technologies](https://ianreppel.org/an-overview-of-apache-streaming-technologies/) (2016-03-12): A comparison table of Apache streaming technologies (Flume, NiFi, Gearpump, Apex, Kafka Streams, Spark Streaming, Storm, Flink, Samza, Ignite, Beam) across dimensions such as delivery semantics, back-pressure, auto-scaling, and in-flight modifications. Flume and NiFi belong in the data collection and single-event processor bucket, while the rest are event stream or complex event processors. The table is not intended to be complete, is unlikely to be entirely accurate, and will almost certainly be out of date in no time.
- [ETL: A Simple Package to Load Data from Views](https://ianreppel.org/etl-a-simple-package-to-load-data-from-views/) (2016-02-29): A simple Oracle package `ETL` loads data from views into tables for single-source/single-target loads, driven by a configuration table with an audit trail. It supports refresh and append modes, lets you purge stale data via a single-column predicate, and combines with Oracle's built-in scheduler so you never need per-object jobs.
- [The Case for Industrial Data Science](https://ianreppel.org/the-case-for-industrial-data-science/) (2016-02-01): Industrial data science is more constrained than the sanitized version taught in courses and MOOCs, and it is time we developed realistic programmes for it. Production quality and product safety are its main drivers, which forces exactly-once event delivery, redundancy, and a tolerance for false positives but not false negatives. It demands on-premise, minimally invasive architectures, repeatable business processes that mature from single-project to process-oriented mode, and broad domain knowledge. It also means navigating corporate politics, where great ideas still meet resistance and sometimes no really means no.
- [Setting up Scala for Spark App Development](https://ianreppel.org/setting-up-scala-for-spark-app-development/) (2016-01-04): A remark.js presentation gets you up and running with Spark, Scala, sbt, and ScalaTest in (almost) no time. Markdown with a touch of CSS beats PowerPoint for code-heavy slides because it separates content from formatting, and the style sheet is freely adaptable.
- [An Overview of File and Serialization Formats in Hadoop](https://ianreppel.org/an-overview-of-file-and-serialization-formats-in-hadoop/) (2015-12-07): When dumping data into Hadoop, the choice of container and serialization format matters. Columnar formats such as ORC and Parquet compress better and suit OLAP, while row-based formats suit OLTP and streaming. Splittability allows files to be processed in parallel, which is why XML and JSON are not ideal. Comparisons by IBM and Hortonworks show Avro and Parquet are best overall for size, reading, writing, and generating statistics.
- [The Fast Track to Julia](https://ianreppel.org/the-fast-track-to-julia/) (2015-09-13): Julia is a programming language designed for technical computing that aims to combine the speed of C with the ease of Python. A cheat page (not a sheet) at ianreppel.org/julia.html covers the salient features, because an HTML page is shareable, updatable, and not limited by paper.
- [Mapping a Value Stream in Neo4j](https://ianreppel.org/mapping-a-value-stream-in-neo4j/) (2015-08-16): A graph database lends itself naturally to modelling value streams and supply chains, which are networks rather than hierarchies. Ian shows how to load supplier, customer, and product data into Neo4j with Cypher and CSV files, then query total yield and cycle time across a product chain with the reduce function. Where Neo4j really shines is retrieving the entire subgraph connected to a source node, a bi-directional traversal that CONNECT BY and recursive common table expressions cannot do.
- [Oracle Date Arithmetic Weirdness](https://ianreppel.org/oracle-date-arithmetic-weirdness/) (2015-07-08): Oracle's INTERVAL literals are colossally stupid: INTERVAL '1' MONTH adds one to the month irrespective of the day, so subtracting it from 31 March raises ORA-01839, and INTERVAL '1' YEAR ignores leap years. Oracle evaluates mathematical expressions in predicates automatically, so there is no need to pre-compute fractions. The advice is to stay away from INTERVAL literals, use simple numbers instead, and write out fractions of days by multiplying by 24, 24*60, or 24*3600 explicitly.
- [What Makes A New York Times Article Popular? (Part 2/2)](https://ianreppel.org/what-makes-a-new-york-times-article-popular-part-2/) (2015-05-22): Feature engineering is the secret sauce that makes some analytical models stand out. Hand-crafted features (SEO-friendly headlines, questions, no-comment recurring pieces, politics, holidays) were beaten by a simple bag-of-words random forest, and in every model word count is the most important predictor: if you happen to have a long piece, you will grab people's attention. Blending models with different features and algorithms improves the AUC further, and the result landed in the top 15% of the Kaggle competition.
- [What Makes A New York Times Article Popular? (Part 1/2)](https://ianreppel.org/what-makes-a-new-york-times-article-popular-part-1/) (2015-05-22): In the MITx Analytics Edge Kaggle competition, Ian built a model to predict which New York Times blog articles become popular, defined as 25 or more comments. Fewer than 16.7% of articles are popular, so a baseline that always predicts unpopular scores 83.2% accuracy. Exploratory analysis shows popular articles have shorter headlines and higher word counts, and that popularity peaks for articles posted around 10PM. Missing news desk, section, and subsection categories are filled in manually rather than imputed.
- [Connecting to Oracle Database VM From A Host](https://ianreppel.org/connecting-to-oracle-database-vm-from-a-host/) (2015-04-26): You can develop against the pre-built Oracle Database 12c VM from your host OS rather than inside the guest. Install Oracle SQL Developer on Ubuntu, set up NAT port forwarding in VirtualBox, and allow port 1521 through the Oracle Enterprise Linux firewall. The standard user pmuser and service name PDB1 then let you connect from the host.
- [Checking Data Type Consistency in Oracle](https://ianreppel.org/checking-data-type-consistency-in-oracle/) (2015-03-29): In large databases, data types diverge across tables over time and cause failures when moving data, because SQL does not understand PL/SQL's %TYPE attribute. A utility procedure identifies type inconsistencies via a view, resolves them to the most prevalent data type, and redefines tables online with DBMS_REDEFINITION so the fix is non-blocking. Ties, values that no longer fit, and custom types are flagged for manual review.
- [An Overview of PL/SQL Collection Types](https://ianreppel.org/an-overview-of-plsql-collection-types/) (2015-03-01): PL/SQL has three collection types (associative arrays, nested tables, varrays) that look similar but differ in persistence, initialization, indexing, and available methods, so choosing the wrong one is a common source of bugs. Varrays suit small, fixed-size, order-preserved data; nested tables suit large, unordered sets of unknown size; associative arrays suit session-scoped lookups. A comparison table condenses the properties that determine which collection fits which job.
- [Why Govern Your Data?](https://ianreppel.org/why-govern-your-data/) (2015-02-01): Data governance is a mindset: it requires a fundamental shift from treating data as a by-product of doing business to managing it top-down as a corporate asset. Dirty data costs businesses up to a quarter of their revenue, and data scientists spend 50-80% of their time cleaning data rather than discovering patterns. More data means more potential issues, and without data governance that potential quickly materializes. A data governance programme needs executive support and typically spreads via the inverted-T model: top-down in one area, then outwards.
- [Oracle SQL and PL/SQL Coding Guidelines](https://ianreppel.org/oracle-sql-and-plsql-coding-guidelines/) (2015-01-04): Coding standards reduce the cost of maintenance because they let database developers on the same team read one another's code more easily. A set of coding conventions for Oracle SQL and PL/SQL, published as an editable Markdown document with a Pandoc build pipeline, so teams can fork, adapt, and generate styled HTML without proprietary tools.
- [Unit Testing PL/SQL Code?](https://ianreppel.org/unit-testing-plsql-code/) (2014-12-21): Unit testing decreases pre-release defects 45-90% while only requiring 15-35% more initial effort, and fixing bugs after release costs 15x more than during development. With Oracle SQL Developer there is no reason to remain in the Dark Ages: unit testing PL/SQL components is easy, free, and fully integrated into the IDE. DBMS_OUTPUT.PUT_LINE does not constitute a test, because it is not easily automated and therefore not repeatable.
- [Tuning Distributed Queries in Oracle](https://ianreppel.org/tuning-distributed-queries-in-oracle/) (2014-12-07): Oracle always runs distributed queries on the local database because it cannot estimate the cost of network traffic. Combining the DRIVING_SITE hint with replacing PL/SQL user-defined functions with built-in functions cut a query from 120 seconds to 12, because the remote database bears the brunt and no longer has to call local functions across the network. The main benefit is in the combination of the driving site hint and the use of built-in rather than remote user-defined functions.
- [Frustration: The Data Governance Sinkhole](https://ianreppel.org/frustration-the-data-governance-sinkhole/) (2014-11-23): Reactive organizations risk losing their smartest data managers when those individuals become frustrated by the lack of attention paid to data quality issues and choose to join organizations that are more responsive. Dull but important tasks such as data profiling are dismissed as extravagances, and developers are not consulted on solutions, which breeds frustration and then boredom. And bored people quit. Without the full engagement of the data experts who do the heavy lifting, an enterprise data strategy seems all but impossible to reach.
- [Searching The Oracle Data Dictionary](https://ianreppel.org/searching-the-oracle-data-dictionary/) (2014-11-09): Finding tables by their column combination in a large database is slow with documentation and Ctrl+F. Oracle's data dictionary is always up to date and queryable, so a PL/SQL package can search for tables and views by column name directly inside the database. The find_tab procedure uses DBMS_SQL to handle an unknown number of search columns at runtime, handling case, separators, and double quotes automatically.
- [Numerical Algorithms: Variational Integrators](https://ianreppel.org/numerical-algorithms-variational-integrators/) (2014-10-26): Classical integrators accumulate discretization errors without bound, which is a huge problem for astronomical and spacecraft simulations. Geometric numerical integrators have bounded errors because they respect the differential geometric structures underpinning the dynamics. Variational integrators derive from the principle of stationary action, and higher-order algorithms come from better approximations to the discretized action, not from magic cancellations of error terms. VarInt is a Maple package that computes these algorithms for you.
- [How to Multiply Across a Hierarchy in Oracle: Performance (Part 2/2)](https://ianreppel.org/how-to-multiply-across-a-hierarchy-in-oracle-part-2/) (2014-10-12): Benchmarks on Oracle 12c show the recursive common table expression (cte) is the fastest way to multiply across a hierarchy, while the nested CONNECT BY solution (mult) is by far the slowest. For tables with more than 1,000 rows, stay away from nested hierarchical queries and go with factored subqueries. The eval and with methods differ little, indicating context switches are not a significant factor for this kind of problem.
- [How to Multiply Across a Hierarchy in Oracle: SQL Statements (Part 1/2)](https://ianreppel.org/how-to-multiply-across-a-hierarchy-in-oracle-part-1/) (2014-09-28): Oracle has no PRODUCT() aggregate function, but you can build one from logarithms: EXP(SUM(LN(col_name))). To multiply a column along each branch of a tree, the CONNECT BY clause needs a nested query that walks back up the hierarchy, while a recursive CTE multiplies the running yield directly in the recursion step. A third approach uses native dynamic SQL to evaluate the SYS_CONNECT_BY_PATH string as an expression.
- [Homelessness: A Problem with Data but Few Solutions](https://ianreppel.org/homelessness-a-problem-with-data-but-few-solutions/) (2014-09-14): About 77.6% of the causes of homelessness are covered by data that is available or at least accessible to governmental agencies, yet precious little is done with it. We know the reasons, we understand the problem, yet we do nothing as societies. Governments could identify individuals at risk of becoming homeless based on information already in their own systems: income, family members, medical information, police records. All governments have to do is act on the data available in their own systems.
- [The Data Quality Doctor](https://ianreppel.org/the-data-quality-doctor/) (2014-08-31): Many companies are stuck in firefighting mode, patching data quality problems with temporary fixes that cause more problems downstream than they really solve. Any solution that does not involve fixing the cause of the problem is only temporary and thus a waste of time. The cure is a simple two-step process: expose the problem and its extent, then excise the cause as close to the source as possible. Don't fight the symptoms when you can cure the disease.
- [Oracle Database Optimization for Developers](https://ianreppel.org/oracle-database-optimization-for-developers/) (2014-08-17): Oracle optimization resources are geared towards DBAs, leaving developers without a one-stop reference for techniques they can apply without DBA privileges. oracle.rtfd.io fills that gap: an open, community-maintained collection of SQL and PL/SQL optimization techniques scoped to what developers can do on their own to make queries run faster and the database more responsive.
- [Data ≠ Information](https://ianreppel.org/data-information/) (2014-08-03): More data does not lead to more information, and a pattern is not an insight. The beer-and-diapers story shows you can make money from a pattern without understanding its cause. Big Data is a paradigm shift, not a revolution, and the value of a megabyte at Netflix is meagre: each additional megabyte per user adds very little value to the company. What I oppose is unfettered and baseless optimism about the promises of Big Data.
- [The Football Fortune Tellers](https://ianreppel.org/football-fortune-tellers/) (2014-07-13): Goldman Sachs and PwC were more clairvoyant than a random selection of winners and losers, but their statistical models were not much better than simply looking at countries who had reached the semi-finals in all World Cups before 2014. Both failed to predict the correct winner, and Goldman Sachs's match results were generally way off. Microsoft's Cortana was spot on for all knockout games, showing it's generally easier to predict the past than it is to predict the future.
- [The Challenges of Data Integration: Data Governance (Part 4/4)](https://ianreppel.org/the-challenges-of-data-integration-part-4/) (2014-07-06): Data governance is the institutionalized programme that deals with all facets of data over the entire life cycle: a company's data governance office is your organization's Ministry of Data. The challenges reduce to two negated implications: system responsibility does not imply content responsibility, and standards do not imply compliance. A common system does not imply identical data structures. Focusing on merely one or two facets of data integration tacitly accepts more risk than you have to.
- [The Challenges of Data Integration: People (Part 3/4)](https://ianreppel.org/the-challenges-of-data-integration-part-3/) (2014-06-22): Data integration projects are tricky because they require dirty data to be 'smuggled' over organizational borders into enemy territory. People live in their own little data worlds and ask 'What's in it for me?', so a business case is the least you need to convince them. Transparency hurts, office politics dominate, and you must manage expectations: data integrity is not something you do once.
- [The Challenges of Data Integration: Project Management (Part 2/4)](https://ianreppel.org/the-challenges-of-data-integration-part-2/) (2014-06-08): Project management is the key to success: data integration without a clear-cut business case is doomed to fail, and active support from all stakeholders including an executive sponsor is essential. Scope creep is by far the largest threat to the project's success, so save non-mandatory changes for future releases. Profile data before the project to size up the effort, and make sure all critical tasks have a bus factor larger than 1.
- [The Challenges of Data Integration: Technical (Part 1/4)](https://ianreppel.org/the-challenges-of-data-integration-part-1/) (2014-05-25): Data integration is never the goal of an organization; the data warehouse is merely a vehicle. The architecture is determined by business requirements and context, and data quality is the single most important item on your to-do list. Data cleansing is time-consuming but absolutely necessary, and my number-one rule is that data quality is always worse than you or any knowledgeable person thinks. Systems are dynamic: data integration is never done.