For Chambers & Counsel
The AI and algorithmic special master.
When a case turns on what a model does, ordinary discovery reaches its limit: the system is non-deterministic, its behaviour is not readable from its code, and its inputs are too large to produce. This page covers what a reference to a neutral is scoped to reach in that situation, and what the order has to say.
The Options
Three ways a court can get technical help, and they are not interchangeable.
Writing for the Brookings Institution in 2019, Melissa Whitney observed that “a variety of strategies are available to educate judges on the AI-related technology at issue in a litigation,” and that “these include the use of science or technology tutorials, court-appointed technical advisers or special masters, and court-appointed experts.” The choice among them is not cosmetic. Each carries different authority, a different relationship to the record, and a different standard of review.
- Special masterFed. R. Civ. P. 53
- Appointed by the court to perform duties the order delegates. Rule 53(a)(1)(C) reaches pretrial matters that cannot be effectively and timely addressed by an available district or magistrate judge — which is what an algorithmic discovery fight usually is. The master answers to the court, and Rule 53(f) fixes how findings are reviewed.
- Court-appointed expertFed. R. Evid. 706
- An expert the court itself appoints, who advises all parties of any findings, may be deposed, may be called to testify, and is subject to cross-examination by any party. The distinction that matters: a Rule 706 expert gives opinion evidence in the case; a Rule 53 master performs a delegated judicial function.
- Technical adviserInherent authority
- A tutor to the court rather than a witness — no testimony, no report in evidence. Useful where the court needs to understand a system rather than resolve a contested fact about it, and correspondingly weaker where the parties genuinely disagree about what the system does.
A tutorial, in Whitney’s phrase, is meant “to transform the courtroom into a classroom” — which is the right instrument when the court needs to understand a technology, and the wrong one when the parties disagree about what a specific system did. For the general distinction between these roles, see discovery referee vs. forensic neutral vs. special master.
The Reference
What the neutral is actually scoped to reach.
An algorithmic reference is not document review with harder documents. Five categories of work recur, and an order that names them is materially cheaper to administer than one that refers “all disputes concerning the model.”
Reproducibility, established rather than assumed
The same prompt can produce different outputs, so no single exhibit establishes what a system does. The neutral designs a test protocol on inputs the parties agree to, runs it enough times to characterise the distribution of outputs, and reports what was observed. This is the work that converts an argument about screenshots into a finding of fact.
Inspection of code, weights, and pipeline under the protective order
Model weights, architectures, and data pipelines are frequently a party's most valuable asset, and neither side will accept the other's expert inside the environment. A neutral examines in a controlled environment and reports only what the court needs, which protects the proprietary material and the record at the same time.
Provenance of disputed training data
Training corpora span many sources, so 'what went in' is an expensive question rather than a simple one. Scoping it is a Rule 26(b)(1) proportionality exercise: which sources, over which period, at what granularity, and to answer which contested issue. A neutral can cost the alternatives before the parties litigate them.
Competing measurements of accuracy, bias, or copying
Where two qualified experts present opposing quantitative accounts, the disagreement is usually in the methodology — the benchmark, the population, the threshold, the metric. A neutral tests the methods and identifies which conclusions the evidence actually supports, which is a narrower and more answerable question than which expert is right.
Authentication of machine-derived exhibits
Rule 901 requires evidence sufficient to support a finding that an item is what its proponent claims. Synthetic audio, video, and documents strain that standard because the ordinary indicia of authenticity are now generable. Proposed Rules 901(c) and 707 address the problem directly and are worth tracking by anyone litigating with machine-derived exhibits.
The Order
Five provisions an AI reference needs that a standard order omits.
Fix the questions, not the tools
The technology moves faster than the docket. An order that enumerates the questions to be answered lets the neutral adapt method to the system; an order that enumerates tools invites a scope fight every time the system changes.
Provide for a controlled testing environment
Inspection of a trained model is not document review. The order should describe where testing occurs, who provisions it, who bears that cost, and what happens to the environment and any derived artefacts at the close of the reference.
Say what a 'result' is before testing begins
Because outputs vary run to run, the order should anticipate how results are characterised and reported — a distribution rather than an instance. Settling this at appointment prevents a dispute about the methodology arriving disguised as a dispute about the findings.
Integrate with the protective order explicitly
Reference the operative protective order and state how the master may handle, image, or retain material under it, and how work product is secured. Rule 53(b)(2)(C) already requires the order to state what is preserved and filed as the record of the master's activities.
State compensation at appointment
Rule 53(b)(2)(E) requires the order to state the basis, terms, and procedure for fixing compensation. Deferring it does not avoid an argument; it schedules one.
The model order for source-code inspection is the closest starting point and pairs each clause with a note on what it does. For the cost mechanics, who pays a special master sets out Rule 53(g) and what the only federal study found about allocation. Where the dispute is as much about production as about the model, see the e-discovery special master.
Sources
- Melissa Whitney, “How to improve technical expertise for judges in AI-related litigation” — The Brookings Institution, November 7, 2019
- Fed. R. Civ. P. 53 (masters); Fed. R. Civ. P. 26(b)(1) (scope and proportionality); Fed. R. Evid. 706 (court-appointed expert witnesses); Fed. R. Evid. 901 (authenticating evidence).
This page describes a practice and the rules that govern it. It is general information, not legal advice, and creates no engagement.
Scoping an Algorithmic Reference
Define the questions before the technology moves.
A short scoping conversation establishes what an algorithmic reference would need to reach and what testing it would require — the inputs to both the scope provision and the compensation provision.