Market Microstructure and High-Frequency Trading

Market microstructure is the study of how trades are executed. Subject matter includes order-book dynamics, the role of intermediaries, the structure of venues, and the regulation that governs them. The discipline coalesced in the late 1980s with Maureen O’Hara’s Market Microstructure Theory (1995) building on Glosten-Milgrom (1985 Journal of Financial Economics 14:71 on bid-ask spreads under adverse selection), Kyle (1985 Econometrica 53:1315 on informed-trader strategic behavior with a market maker and noise traders), and Roll (1984 Journal of Finance 39:1127 on the implied bid-ask spread from transaction-price serial covariance). It became operationally critical when US equity markets fragmented post-Reg NMS (2005) and tick-by-tick latency arbitrage scaled into a multi-billion-dollar industry. By 2024, electronic market makers handled approximately 60% of US equity volume and HFT firms collectively had become the de facto liquidity providers for both equities and futures.

Market structure: call vs continuous, dealer vs auction

A call market crosses all orders at a single clearing price set periodically (NYSE open/close auctions, the daily Tokyo Stock Exchange afternoon open auction itayose). A continuous market matches incoming marketable orders against the resting book in real time (the default model for equities since the mid-1990s). Dealer (quote-driven) markets — historical Nasdaq pre-1997, OTC corporate bonds, FX spot, swap dealer markets — have designated dealers posting bid/ask quotes and risking principal capital. Auction (order-driven) markets — NYSE specialist post-1869, modern continuous limit-order books — match arriving orders directly without a dealer principal. Hybrid models (post-2006 NYSE Pillar, Cboe BZX with periodic auctions, IEX with the 350-microsecond speed bump) combine continuous trading with periodic auctions at open and close and with structural friction.

The closing auction has become the dominant single liquidity event of the US day. As of 2024 roughly 10-12% of US listed volume crosses in the closing call (NYSE Closing Auction, Nasdaq Closing Cross), driven by passive index-fund rebalancing and the proliferation of MOC (market-on-close) institutional execution algorithms. The opening auction (NYSE 09:30 ET, Nasdaq Opening Cross) handles roughly 2-3% of daily volume. Cboe Closing Cross, MEMX Closing Auction, and IEX Closing Auction (launched October 2023) compete for closing-auction volume, with Cboe and IEX offering price improvement and reduced impact relative to the primary listing exchange’s official close.

Order types

Market order — execute immediately at best available price; price uncertain. Limit order — execute only at specified price or better; may not execute (becomes part of the resting book). Stop order — becomes a market order when last trade reaches the stop. Stop-limit — becomes a limit order on stop trigger. Marketable limit — limit order priced through the opposite side that executes immediately like a market order but caps slippage.

Time-in-force qualifiers:

• IOC (immediate-or-cancel) — execute the marketable portion and cancel the rest;
• FOK (fill-or-kill) — execute the entire size immediately or cancel entirely;
• GTC (good-til-cancelled) and GTD (good-til-date) — persist across sessions;
• Day — expire at end-of-day if unfilled;
• ATC (at-the-close) and ATO (at-the-open) — participate only in closing or opening auction.

Hidden orders rest invisibly; iceberg / reserve orders display only a portion (e.g., display 100 shares of a 10,000-share order, replenish on execution). Peg orders track the National Best Bid and Offer (NBBO) — primary peg (locked to same-side NBBO), midpoint peg (between NBB and NBO), market peg (opposite-side NBBO). Discretion adds an undisplayed price-improvement willingness above the displayed limit — common in dark pools. D-Limit (IEX, introduced October 2020) — a discretionary limit order that retreats from the NBBO if IEX’s signal detects an imminent quote update, designed to protect liquidity providers from latency arbitrage.

Order book dynamics

The limit order book is the price-time priority queue of resting bids and offers. Time priority — within a price level, the earlier order executes first. Price priority — better-priced orders execute before inferior ones. Pro-rata allocation (used by some futures and Treasury venues including BrokerTec for Treasuries, CME Eurodollars, ICE Brent partially) splits incoming volume in proportion to displayed size, removing time priority and reducing the queue-jumping advantage of HFT.

Order book imbalance (sum of bid volume / sum within k levels) is a short-horizon predictor of price moves widely exploited by HFT — Cartea-Jaimungal (2016 Mathematics and Financial Economics 10:339) characterized its predictive power formally.

Cont-Stoikov-Talreja (2010 Operations Research 58:549) modeled the order book as a queueing system with birth-death dynamics. Avellaneda-Stoikov (2008 Quantitative Finance 8:217) derived optimal market-making bid/ask spreads under inventory aversion using a reservation price plus inventory skew. Cartea-Jaimungal-Penalva Algorithmic and High-Frequency Trading (2015 Cambridge University Press) is the standard graduate text. Almgren-Chriss (2001 Journal of Risk 3:5) provided the optimal-execution framework trading market impact against timing risk that underlies VWAP and IS algorithms — the linear permanent + linear temporary impact model with quadratic execution cost.

Dark pools and ATSs

Alternative trading systems (ATSs) execute matched volume without pre-trade displayed quotes, regulated under SEC Reg ATS (1998) and the enhanced Reg ATS-N (effective January 7, 2019).

Major dark pools:

• Liquidnet (founded 1999 by Seth Merrin, block-size focus, acquired by TP ICAP 2021 for $700M);
• ITG POSIT (Investment Technology Group, founded 1987 as MatchPoint, acquired by Virtu 2019 for $1.0B);
• UBS PIN (UBS Price Improvement Network — frequently the largest US dark pool by volume);
• JPM-X (JPMorgan);
• Credit Suisse Crossfinder (post-2009 expansion);
• Goldman Sigma X;
• MS Pool (Morgan Stanley Trajectory Cross);
• BIDS Trading (Cboe BIDS, founded 2007 by major broker consortium, acquired by Cboe 2021).

Dark venues handle approximately 15-18% of US consolidated equity volume in 2024 per FINRA OTC ATS Transparency Data, with another ~15% executing on Single Dealer Platforms (SDPs) and retail wholesalers (Citadel Securities, Virtu Americas, G1, Two Sigma Securities, UBS).

Dark pools were tightened by Reg ATS-N (2018) requiring detailed Form ATS-N disclosure after a wave of enforcement:

• Barclays LX 2014 $70M (misrepresentation of HFT activity in the pool);
• Credit Suisse Crossfinder 2016 $84M (sub-penny pricing violations and venue-routing disclosures);
• ITG POSIT 2015 $20M (proprietary trading conflicts with customer order flow);
• Deutsche Bank SuperX 2018 $40M.

ECN history and venue fragmentation

Electronic communication networks (ECNs) replaced telephone dealer markets for Nasdaq in the late 1990s.

Island ECN (Joshua Levine 1997) revolutionized order matching with a published API and ~10-millisecond round trips. Island merged with Instinet 2002 forming INET; acquired by Nasdaq 2005 forming the matching engine Nasdaq still uses.

Archipelago Exchange (ArcaEx, founded 1996 by Gerald Putnam in Chicago) merged with NYSE March 2006 (the NYSE Group), becoming NYSE Arca. BRUT ECN (acquired by Nasdaq 2004). BATS Trading (Better Alternative Trading System, founded 2005 by Dave Cummings in Kansas City) became Bats Global Markets, IPO 2016, acquired by Cboe Global Markets 2017 — now operating Cboe BZX, BYX, EDGA, EDGX. Direct Edge (founded 2007, merged with BATS 2014, contributing the EDGA and EDGX venues).

The major US equity venues 2024:

• NYSE Group (NYSE, NYSE Arca, NYSE American, NYSE National, NYSE Chicago);
• Nasdaq (Nasdaq, Nasdaq BX, Nasdaq PSX);
• Cboe (BZX, BYX, EDGA, EDGX);
• IEX Investors Exchange (founded 2012 by Brad Katsuyama, exchange status 2016, the 350-microsecond speed bump);
• MEMX Members Exchange (launched September 2020, backed by Citadel-Virtu-Goldman-Fidelity-Schwab-BofA-JP Morgan et al.);
• MIAX PEARL Equities (2020, Miami International Holdings);
• LTSE Long-Term Stock Exchange (2020, Eric Ries-led venture targeting long-horizon listings).

The 16 SEC-registered exchanges, the ~30 active ATSs, the wholesalers, and the SDPs together create one of the most fragmented equity-market structures globally; the median US listed stock trades on 30+ venues simultaneously.

Reg NMS and the rules of the road

Regulation National Market System (SEC Reg NMS, adopted June 2005, compliance July 2007) consists of:

• Rule 611 (Order Protection Rule, “trade-through rule”) — venues must route around to a protected quote at a better price; this drove fragmentation by making every venue’s BBO compete for protected-quote status.
• Rule 610 (Access Rule) — fair access to quotes, cap on access fees at 30 mils ($0.003/share); the rebate-fee model uses this cap.
• Rule 612 (Sub-Penny Rule) — minimum tick $0.01forstocks\ge$1.00, sub-penny allowed below.
• Rule 600 — definitions including the consolidated NBBO, protected quote, automated quotation.
• Rule 613 — Consolidated Audit Trail (CAT, adopted 2012).

Reg ATS (1998) regulated dark pool operations; Reg ATS-N (2018) added Form ATS-N disclosure.

Reg SHO (2005) governs short selling:

• Rule 200 — Order marking (long, short, short-exempt);
• Rule 201 — Alternative uptick rule / “circuit-breaker uptick” (October 2010) replacing the original uptick rule (Rule 10a-1, 1938, repealed July 2007) — short-sale price test triggers when a stock falls 10% intraday from prior-day close;
• Rule 203 — Locate requirement (broker must reasonably believe shares are available to borrow before executing a short sale);
• Rule 204 — T+2 close-out (now T+1 after May 28, 2024 settlement-cycle shortening).

The August 2023 SEC adoption of Rule 13f-2 added monthly short-sale activity reporting (effective January 2025) and Form SHO disclosure.

MiFID II (Markets in Financial Instruments Directive II, EU, effective January 3, 2018) requires best execution reporting (RTS 27/28), double-volume caps on dark trading (4% per venue, 8% market-wide, since superseded by Single Volume Cap mechanism 2023), tick-size regime mandating common tick across EU venues, and transaction reporting via APAs (Approved Publication Arrangements). The UK post-Brexit MiFID II refit (HM Treasury 2023 Wholesale Markets Review) loosened the dark-trading caps and the share-trading obligation.

Origins of modern HFT

Quantitative trading predates electronic markets. Renaissance Technologies (founded 1982 by James Simons in East Setauket NY) was running its Medallion strategies on mainframes by the late 1980s, exploiting daily-and-intraday signals. Modern microsecond-latency HFT emerged after Reg NMS (2005) created fragmented competing venues whose quotes had to be reconciled.

Leading HFT and electronic market-making firms (founding dates and key facts):

• Tradebot Systems — founded 1999 by Dave Cummings, Kansas City; spun off BATS Trading in 2005.
• GETCO (Global Electronic Trading Company) — founded 1999 Chicago; merged with Knight Capital 2013 forming KCG Holdings; acquired by Virtu 2017 for $1.4B.
• Citadel Securities — founded 2002 by Ken Griffin as Citadel’s market-making arm; today the largest US retail wholesaler, executing approximately 35-40% of US listed-equity retail flow; payment-for-order-flow (PFOF) recipient from Robinhood, Schwab, E*TRADE.
• Virtu Financial — founded 2008 by Vincent Viola; IPO 2015 NASDAQ:VIRT.
• Jane Street Capital — founded 2000 by Tim Reynolds and Marc Gerstein, originally Susquehanna spinoff; major ETF market maker; reported ~$10B net revenue 2023.
• Optiver — Amsterdam-based, founded 1986 by Johann Kaemingk; major options market maker globally.
• IMC Trading — Amsterdam, founded 1989; options and ETF market making.
• DRW Holdings — Don R. Wilson, founded 1992 Chicago; futures and crypto market making.
• Susquehanna International Group (SIG) — founded 1987 Bala Cynwyd PA by Jeff Yass; one of the largest options market makers.
• Jump Trading — founded 1999 Chicago by Bill DiSomma and Paul Gurinas; multistrat HFT + DeFi/crypto via Jump Crypto.
• XTX Markets — founded 2015 London by Alex Gerko, ex-Deutsche Bank; ML-driven FX and equity market making; >$1B annual revenue 2023.
• GTS (Global Trading Systems) — founded 2006 NYC; NYSE DMM (Designated Market Maker) since 2016 for >100 of the most-active NYSE issues.
• Hudson River Trading — founded 2002 NYC; quantitative market making across asset classes.
• Tower Research Capital — founded 1998 by Mark Gorton; affiliated with Limewire historically; HFT in equities and futures.
• Two Sigma Securities — founded 2009 as market-making arm of the Two Sigma hedge-fund parent.

HFT strategies

Electronic market making — post simultaneous bids and offers, earn the spread, manage inventory risk by skewing quotes when long or short. Modern bid-ask spreads on liquid US large-caps are 1-2 ticks ($0.01-$0.02) routinely. Citadel Securities, Virtu, Jane Street, Two Sigma Securities, GTS, Hudson River Trading, and Tower Research are the dominant US equity wholesalers. Off-exchange wholesalers — Citadel Securities, Virtu, G1, Two Sigma Securities, UBS, Wolverine — handle the bulk of retail-broker PFOF flow, internalizing at NBBO or better with price improvement averaging 0.05-0.20 cents per share.

Statistical arbitrage — pairs trading (cointegrating stock pairs, mean-reverting spreads) introduced systematically at Morgan Stanley by Nunzio Tartaglia’s group in the mid-1980s. Gerry Bamberger and David Shaw worked on the original Tartaglia group; Shaw later founded D.E. Shaw 1988. Modern stat-arb practitioners:

• Renaissance Technologies Medallion (~66% gross annualized since 1988 per public filings; only open to employees since 1993);
• D.E. Shaw (multistrat, ~$60B AUM 2024);
• Two Sigma (founded 2001 by John Overdeck-David Siegel-Mark Pickard);
• Citadel (multistrat hedge fund, ~$65B AUM 2024);
• Bridgewater Associates (Ray Dalio, Pure Alpha) — closer to global-macro than HFT, the line blurs.

Marcos López de Prado Advances in Financial Machine Learning (2018) codifies modern quant practices including triple-barrier labeling, meta-labeling, and combinatorial purged cross-validation.

Latency arbitrage — exploit time differences in price updates across venues. When Cboe lifts the offer in Chicago at t=0, the trade-through-protected quote at NYSE Arca in Mahwah NJ will update only after the consolidated tape catches up. An HFT with private direct feeds and the Chicago-NJ microwave link can act on the Chicago print before the SIP reflects it. This is the controversy chronicled by Michael Lewis Flash Boys (2014) and motivated the founding of IEX with its 350-microsecond speed bump.

Index arbitrage — exploit basis between cash index baskets and SPX futures (CME ES, MES, also Cboe SPX options). ETF arbitrage — APs (authorized participants) create/redeem ETF shares against underlying baskets when ETF price diverges from NAV by more than the basis (typically a few basis points for liquid ETFs like SPY, QQQ, IWM). Basis trading — cash Treasury vs Treasury futures spread, central to Treasury market liquidity but blamed for the March 2020 dash-for-cash dislocations when leveraged basis-trade unwinds amplified Treasury-market dysfunction.

Latency: from milliseconds to nanoseconds

Speed-of-light optical fiber between the CME Group data center in Aurora, Illinois (the former Cermak campus) and Equinix NY4 in Secaucus NJ runs roughly 6.5 ms one-way on the best fiber path. The Nasdaq matching engine is at Carteret NJ and the NYSE matching engine is at Mahwah NJ — both in northern New Jersey, distinct sites. Microwave point-to-point networks complete the Chicago-NJ leg in approximately 4.0-4.5 ms — nearly the great-circle light-speed limit of ~4.0 ms.

Microwave network operators:

• Anova Financial (acquired by Zayo, then Zayo acquired by EQT/Digital Colony 2020);
• McKay Brothers (founded 2009 by Bob Meade and Stéphane Tyč; Chicago-NJ link operational 2012);
• Quincy Data (also McKay Brothers, focused on data distribution);
• Custom Connect (Netherlands, NY-London-Frankfurt links).

Hollow-core fiber (Lumenisity, acquired by Microsoft 2022; NANF nested-antiresonant nodeless fiber) propagates light through air-filled cores at ~99.7% c versus silica’s ~67% c, giving roughly 33% latency reduction over equivalent silica routes.

The Chicago triangle (CME Aurora ↔ Equinix LD4 Slough ↔ Equinix CH1 Chicago) and the trans-Atlantic Hibernia Express cable (completed September 2015, NY-London ~58.95 ms RTT) and ASN Apollo South are intercontinental analogues. The Arctic Connect / Polar Express and Far North Fiber projects target shorter Asia-Europe routes via the Arctic.

Inside the data center, cross-connects between trading firm cages and exchange matching engines are typically 1-3 microseconds. FPGAs (Xilinx Virtex UltraScale+, Intel Stratix 10, Achronix Speedster7t) implement market-data parsing and order generation in tens to low-hundreds of nanoseconds. Some firms quote 80-120 ns tick-to-trade — the time from incoming market-data packet to outgoing order packet at the NIC. NIC-bypass kernel-bypass networking (Solarflare Onload, Exablaze, Mellanox VMA) shaves microseconds vs standard kernel TCP/IP.

Exchange technology

CME Globex (Chicago Mercantile Exchange’s electronic platform, launched 1992) handles CME, CBOT, NYMEX, COMEX futures with sub-100-microsecond matching since the Globex 2.0 rewrite. Nasdaq INET (acquired 2005, descended from Island) is the matching engine for Nasdaq, NLX, and is licensed to other venues globally. NYSE Pillar (rolled out 2017) consolidates NYSE, NYSE Arca, NYSE American, NYSE National onto one platform with sub-30-microsecond match. ICE iMpact (Intercontinental Exchange) runs ICE Futures US, ICE Futures Europe, NYSE Bonds. London Stock Exchange Millennium Exchange (built by MillenniumIT, acquired 2009; deployed 2011) achieves sub-100-microsecond matching. Deutsche Börse Eurex T7 (2013, with the Eurex Clearing C7 system) handles Eurex futures and options. Johannesburg Stock Exchange uses MillenniumIT. Australian Securities Exchange uses ASX Trade (NASDAQ OMX Genium INET) and ASX Trade24 for futures. Tokyo Stock Exchange Arrowhead (launched January 2010, 2015 upgrade to sub-100-microsecond matching, Arrowhead 4.0 2024). Hong Kong Exchange OCG-C/OCG-D (Orion Central Gateway). National Stock Exchange of India NEAT and NSE EMS.

Colocation pricing: a half-rack cabinet at NYSE Mahwah, Nasdaq Carteret, or CME Aurora runs roughly $5,000-$20,000/month base, plus power ($1-3k/kWpermonth),cross-connects($300-2000/month each), and data-feed fees. Top-tier rooms with shortest fiber drops sell at premium. SEC v Bats Global Markets (2019) and the related litigation around colocation fee fairness shaped Reg NMS Rule 610 fee-cap interpretation.

Market data: SIPs vs direct feeds

The Consolidated Tape Association (CTA) administers the CTS (Consolidated Trade System for NYSE-listed) and CQS (Consolidated Quote System). “Tape A” SIP carries NYSE-listed equities, “Tape B” carries NYSE-Amex/regional, and “Tape C” carries Nasdaq-listed equities (under the Unlisted Trading Privileges Plan, UTP). These Securities Information Processors aggregate all venues into the official NBBO published with measurable latency — historically tens to hundreds of microseconds slower than colocated direct feeds. Reg NMS Rule 600(b)(82) defines the NBBO based on the SIP, creating the latency-arbitrage opportunity: a trader with direct feeds knows the true best price before the SIP does.

Direct feeds (proprietary, non-aggregated, full depth):

• Nasdaq TotalView-ITCH (full depth-of-book, atomic order-by-order events);
• Nasdaq BX ITCH and Nasdaq PSX ITCH;
• NYSE Integrated Feed and NYSE OpenBook Ultra;
• Cboe Book Viewer / PITCH for BZX/BYX/EDGA/EDGX;
• ICE Best for ICE futures and equities;
• CME MDP 3.0 (Market Data Platform) for CME/CBOT/NYMEX/COMEX futures.

Vendor-consolidated feeds:

• Refinitiv (formerly Thomson Reuters) Tick History;
• ICE Data Services Consolidated Feed;
• Bloomberg BPipe;
• Activ Financial;
• QuantHouse (acquired by S&P Global 2021).

Historical tick data for backtesting from Refinitiv, Bloomberg, IEX market-data API (free for academic use), NYSE TAQ for academic research, and modern cloud-native vendors Algoseek, Polygon, Databento, and Kaiko (crypto).

The SEC’s proposed Reg NMS amendments to expand SIP content were adopted December 2020 (Market Data Infrastructure Rule) — expanding to include depth of book to five levels, auction information, and odd-lot quotes — and partially struck down by the DC Circuit (Nasdaq v SEC, 2022) over the governance changes; subsequent rulemakings continue 2023-2025.

Execution algorithms and SOR

Smart order routing (SOR) splits parent orders across venues to minimize execution cost.

Sell-side institutional execution platforms:

• Goldman Sachs SIGMA X (and SIGMA X Plus);
• Credit Suisse AES (Advanced Execution Services), the modern UBS AES after the 2023 merger;
• JPMorgan Aqua and AlgoX;
• Citi Lava;
• Morgan Stanley QSI / Speedway;
• Bank of America Instinct;
• Barclays LX (relaunched after the 2014 NY AG settlement and Reg ATS-N compliance);
• Bernstein BLEND and BLEND-X;
• Liquidnet H2O (algo-driven block crossing).

Standard execution algorithms:

• VWAP (volume-weighted average price) — slice the day by historical volume profile;
• TWAP (time-weighted) — equal slices over a horizon;
• POV (percent of volume) — track 5-25% of real-time volume;
• IS (implementation shortfall) — Perold 1988 Journal of Portfolio Management 14:4 — minimize total cost vs decision price using Almgren-Chriss optimal execution;
• Liquidity Seeker / Sniper — aggressive dark-liquidity hunters that probe ATSs and SDPs;
• Close (MOC algorithms) — target the closing auction with structured ramp toward 16:00;
• Pairs algos — execute a long/short basket while maintaining hedge ratio;
• Portfolio algos — execute a multi-name basket minimizing tracking error and impact.

Almgren-Chriss (2001 Journal of Risk 3:5) and its extensions (Almgren 2003 Applied Mathematical Finance 10:1; Almgren-Lorenz 2007) underlie most institutional implementations.

Crashes and incidents

Flash Crash, May 6, 2010. The Dow Jones Industrial Average dropped approximately 1,000 points (9%) in minutes around 14:42 ET, recovering most losses by 15:08. SEC-CFTC joint report (September 30, 2010) attributed the trigger to a $4.1B sell program in E-mini S&P futures by Waddell & Reed using a Barclays SELL algorithm pegged to 9% of volume without price or time limits. The cascade vacuumed liquidity across cross-asset markets; some stocks printed at penny prices via stub quotes.

Regulatory response:

• Market-wide single-stock circuit breakers (Limit Up-Limit Down LULD, effective May 2012);
• Elimination of stub quotes;
• Expanded existing market-wide circuit breakers (Rule 80B: 7%/13%/20% drops on S&P 500 from the previous close, with levels updated 2012);
• Clearly Erroneous Execution policy updates by exchanges.

Knight Capital, August 1, 2012. Knight deployed an order-routing system update that re-activated an obsolete code path (Power Peg) on seven of eight production servers. In 45 minutes Knight bought high and sold low across 154 NYSE-listed stocks, accumulating $7Binlong/shortpositionsand$440M in losses, requiring an emergency rescue investment that diluted shareholders ~70%. The SEC settlement (October 2013) imposed a $12M fine and detailed remediation requirements that became the template for SEC Reg SCI (Systems Compliance and Integrity, adopted November 2014, compliance November 2015). Knight was acquired by GETCO November 2013 forming KCG Holdings; KCG acquired by Virtu 2017.

August 24, 2015 ETF crash. The Chinese-yuan devaluation triggered S&P futures limit-down overnight. At the US open, NYSE LULD halts on 1,278 ETFs and ETPs caused massive price dislocations, with some ETFs trading at >35% discounts to NAV momentarily. SPDR S&P 500 ETF SPY traded as low as $189againstNAVnear$197. Cause: NYSE Rule 48 invocation, fragmented opening procedures, ETF AP arbitrage breakdown when underlying-basket prices were unreliable.

October 15, 2014 Treasury flash rally. The 10-year Treasury yield fell 16 bp in 12 minutes and reversed without identifiable news. The joint Treasury-Fed-SEC-CFTC report (July 13, 2015) attributed the move to PTFs (principal trading firms — HFT in Treasuries) shifting hedging behavior and an order-book imbalance feedback loop with little net new information. Catalyzed the move toward Treasury market reform and the FIMSAC (SEC Fixed Income Market Structure Advisory Committee). March 2020 Treasury dysfunction during the COVID-19 dash-for-cash episode reinforced this — the Fed had to launch unprecedented Treasury purchases to restore liquidity.

Spoofing prosecutions:

• Michael Coscia (Panther Energy Trading) was the first convicted under the Dodd-Frank anti-spoofing statute (CEA §4c(a)(5)), sentenced to three years in 2016;
• Navinder Sarao (UK), the “Hound of Hounslow,” pled guilty 2016 to spoofing the E-mini S&P with substantial contribution to the 2010 Flash Crash; received one year home detention 2020 after cooperation;
• JPMorgan precious-metals desk paid $920M in 2020 to settle CFTC-DOJ-SEC spoofing charges; two former traders (Michael Nowak and Gregg Smith) convicted August 2022;
• Bank of America Merrill Lynch precious metals $25M CFTC settlement 2021;
• Mitsubishi UFJ MUFG precious-metals spoofing $43M settlement 2023;
• HSBC Holdings spoofing $75M settlement 2023.

Machine learning and alternative data

Renaissance Medallion’s edge has long been signal mining. David Magerman, Robert Mercer, Henry Laufer applied speech-recognition-style HMMs to market data starting in the 1990s — Mercer was previously at IBM Research’s speech-recognition group with Frederick Jelinek. Modern firms (Two Sigma, Citadel, DE Shaw, Jane Street, Hudson River Trading, XTX Markets, G-Research) employ thousands of researchers building ML signal stacks.

Alternative-data vendors:

• Quandl (acquired Nasdaq 2018);
• 1010data;
• Yipit Data;
• Eagle Alpha (data marketplace);
• Predata;
• Thinknum;
• RavenPack (news sentiment);
• SimilarWeb (web traffic);
• Earnest Research (acquired Vista 2021 — credit-card panel);
• Second Measure (acquired Bloomberg 2021 — credit-card panel);
• Orbital Insight (geospatial);
• Genscape (energy real-time monitoring, acquired Wood Mackenzie 2019).

Data types and providers:

• Satellite imagery for parking lots (RS Metrics, Orbital Insight);
• Credit-card transaction panels (M Science, Second Measure, Earnest Research, Facteus);
• Web scraping (Yipit, Thinknum, SimilarWeb);
• Shipping AIS (MarineTraffic, Windward, Spire Maritime);
• Oil tanker inventories and crude floating storage (Kayrros, Vortexa, OilX, Ursa Space);
• Patent filings and IP analytics (PatSnap, IFI CLAIMS);
• Geolocation foot traffic (Placer.ai, Foursquare/Unfolded).

JPMorgan launched IndexGPT (2024) and ChatGPT-style internal LLMs for research. Goldman Sachs internal generative-AI usage spans research summaries to code generation. Morgan Stanley AI @ Morgan Stanley (with OpenAI partnership) handles wealth-advisor knowledge retrieval. The 2023 EU AI Act and SEC proposed Conflicts-of-Interest Rule on Predictive Data Analytics (Rule 211(h)(2)-3, proposed July 2023) constrain ML use in retail-facing recommendations.

Regulation: CAT, OATS, T+1

Consolidated Audit Trail (CAT, SEC Rule 613, adopted July 2012). The largest financial-market surveillance system ever built — every order, route, modification, cancel, and execution across US equities and options reported by every broker-dealer to a central repository operated by FINRA CAT LLC. Replaced OATS (Order Audit Trail System, NASD Rule 6950, 1998) which covered Nasdaq-only. CAT NMS Plan equity reporting began phased rollout 2018; full options reporting and full small-broker compliance reached 2022; OATS retired September 1, 2021. CAT funding was litigated through 2022-2024 with industry pushback over the $200M+/yr cost; CAT fee allocation under Rule 613(d) has been a continuous regulatory friction.

T+1 settlement. The SEC adopted final rules (February 15, 2023) shortening US securities settlement from T+2 to T+1, effective May 28, 2024 — moving in lockstep with Canada and Mexico (both also T+1 from May 27, 2024). Same-day affirmation requirements (DTCC ITP — Institutional Trade Processing platform) and CCP margin reductions followed. T+0 / atomic settlement (DLT-based) is being studied but is hampered by the loss of multilateral netting benefits at the NSCC; the DTCC’s Project Ion (initially T+1 demo, expanding to T+0 prototype) and the broker-driven Paxos digital settlement pilot are ongoing.

Best-execution reporting:

• MiFID II RTS 27 (venue execution-quality) and RTS 28 (broker top-five-venue) — suspended 2021 by ESMA pending review;
• SEC Rule 605 (market-order quality) — amended March 2024 to expand scope to all NMS stocks and add new metrics;
• SEC Rule 606 (broker order-routing disclosure) — expanded 2018 with detailed PFOF and venue-routing tables;
• MiFIR transaction reporting (~5 billion reports/yr to NCAs in 2023).

The SEC’s Equity Market Structure proposals of December 14, 2022 — Reg Best Execution, Order Competition Rule, Tick Size and Access Fee, Rule 605 enhancements — are the most ambitious overhaul in two decades. Status as of 2024-2025:

• Tick Size and Access Fee Rule adopted September 2024 (variable ticks below $0.01 for tick-constrained stocks, lower access-fee cap);
• Rule 605 amendments adopted March 2024;
• Order Competition Rule (proposed auction of retail orders) remains contentious and not adopted;
• Reg Best Execution (codifying duty for broker-dealers parallel to FINRA Rule 5310) re-proposed and adopted in modified form.

Adjacent

• derivatives-and-quant-finance
• derivatives-and-instruments-catalog
• stochastic-calculus
• fpga-and-hardware-acceleration
• corporate-finance-and-markets
• game-theory