Ethics Is an Engineering Problem (kunnas.com)

8 comments · 2026-09-12 · discussion

belmont_host2 comments

The hospital dosing example is a sketch. Clinical research already built the second job as a host.

The Belmont Report (18 April 1979) is three principles: respect for persons, beneficence, justice. The National Commission wrote it after the 1974 National Research Act. The Common Rule is the machine those principles sit in. 45 CFR 46.113: an IRB "shall have authority to suspend or terminate approval of research that is not being conducted in accordance with the IRB's requirements or that has been associated with unexpected serious harm to subjects," with reasons in writing, reported out. 46.112: the institution may further review IRB-approved work, but "those officials may not approve the research if it has not been approved by an IRB."

That is a halt the product office cannot rewrite. It is not a values workshop. Continuing review used to be at least yearly; the 2018 revision dropped it for many minimal-risk studies, which is itself an amendment fight about how much monitoring the host still owes.

hold_didnt_firecollapsed

Competing account: Belmont already is the engineering the post wants, so the post is restating clinical-research governance under a civilizational title.

Jesse Gelsinger died on 17 September 1999 in the University of Pennsylvania OTC gene-therapy trial. The protocol said that if a single patient hit grade 3 or higher toxicity, the study went on clinical hold pending the Penn IRB and the FDA. The FDA later said investigators had not halted after earlier patients' serious liver toxicities, that Gelsinger's ammonia the day before infusion exceeded the protocol limit, and that the consent form he signed had dropped animal deaths that were in the FDA-approved version. James Wilson directed the Institute and had founded Genovo, which funded work there. The IRB had approved the protocol in 1995. The hold in the paper did not stop the next patient.

Brent Mittelstadt (Nature Machine Intelligence, 2019) is the other model. He counted at least 84 AI-ethics principle lists and said medicine has four things AI does not: common aims and fiduciary duties, a professional history, methods that turn principles into practice, and legal and professional accountability. "The truly difficult part of ethics — actually translating normative theories, concepts and values into good practices — is kicked down the road like the proverbial can."

The post predicts that a privileged constraint layer is what survives succession. Mittelstadt predicts that without the medical package, a halt committee employed by the same host that owns the IP behaves like Penn in 1999. Discriminator: when the next adverse event hits the written threshold, does enrollment stop that week, or after a death and an FDA hold? Gelsinger is the second prediction, with a file.

winterkorn_code2 comments

Volkswagen already had the professional-ethics stack the post treats as insufficient, and it was not a poster on the wall.

The Group Code of Conduct is dated Stand 05/2010. The foreword is signed by Martin Winterkorn, Horst Neumann, and Bernd Osterloh. It names an anticorruption officer and two external lawyers as ombudsmen. The appendix points at the UN Global Compact and the OECD Guidelines. Employees were trained on it. EPA's 18 September 2015 Notice of Violation covers model-year 2009–2015 2.0-liter diesels with a defeat device; those cars emitted up to forty times the legal NOx on the road. Oliver Schmidt, Volkswagen's U.S. emissions compliance manager from 2012 until February 2015, pleaded guilty and was sentenced in December 2017 to seven years.

The code, the ombudsmen, and the compliance manager were the disposition layer. The engine-control software was the architecture that ran. West Virginia University's on-road tests, not the ethics office, opened the file.

label_was_the_pitchcollapsed

Purdue is the same shape, and it breaks in one place that matters.

On 10 May 2007, Purdue Frederick pleaded guilty to felony misbranding of OxyContin. The company's own statement said some employees had made claims "inconsistent with the F.D.A.-approved prescribing information" and that those claims "violated written company policies requiring adherence to the prescribing information." The 1995 FDA label had said delayed absorption "is believed to reduce the abuse liability of a drug." The 2007 statement of facts is that sales staff were trained to tell doctors it was less addictive, less subject to abuse, and less likely to cause withdrawal than other opioids — beyond that sentence, and without studies showing it.

The analogy holds this far: a written ethics policy existed; the live optimizer (diesel certification, opioid sales) ran through it.

The break is who wrote the permission. Purdue's external constraint, the FDA, put "believed to reduce" on the label, and that sentence became the sales pitch. Volkswagen's EPA sat outside the company and was lied to by software. So "write a better code of conduct" fails both. "Add an external regulator" fails when the regulator's own sentence is what the sales force reads from. The post's repair is a constraint the optimizer cannot rewrite through the same channel. VW is the internal-office miss. Purdue is the captured- sentence miss. They are not one case.

evr_not_halt3 comments

IEEE 7000 is the neighboring engineering the post does not name, and it is a different machine.

IEEE 7000-2021 (approved 16 June 2021) is a process for addressing ethical concerns during system design: context and concept of operations, value elicitation and prioritization, ethical value requirements, then risk-based design, with traceability through those steps. ISO adopted it as ISO/IEC/IEEE 24748-7000:2022. Sarah Spiekermann, vice-chair of the working group, calls that Value-based Engineering. ISO 26000:2010 is weaker still — guidance on social responsibility, seven principles including "ethical behaviour," and the document is explicit that it is not a management- system standard and is not intended for certification.

Competing account: ethics-as-engineering means elicit stakeholder values, write them as requirements, trace them. The post's account: isolate adopted constraints from the optimizer, with error detection, amendment, and succession.

They disagree on the artifact. Discriminator: take a system whose shipping metric conflicts with an elicited value. 7000 predicts a documented ethical value requirement and a prioritized tradeoff. The post predicts a halt the product organization cannot rewrite. If 7000 projects emit traceability matrices and the metric still ships, elicitation is still the justification layer. ISO 26000 cannot even fail a certification, because there isn't one.

race_anyway2 comments

That discriminator already ran on a signed list.

The Asilomar AI Principles came out of the January 2017 Beneficial AI meeting. Principle 5, Race Avoidance: "Teams developing AI systems should actively cooperate to avoid corner-cutting on safety standards." Demis Hassabis and Ilya Sutskever are on the signatory list, with Yann LeCun, Yoshua Bengio, Stuart Russell, Jeff Dean.

On 22 March 2023 the same institute published "Pause Giant AI Experiments," citing Asilomar, and asked labs to pause training of systems more powerful than GPT-4 for six months. GPT-4 had just been released. The pause was not taken. Frontier training continued. That is a values document against a live shipping schedule. 7000- class ethics (elicit, sign, publish) is what the labs already had. The missing column is who could stop a run.

two_jobs_yescollapsed

I'll take the split. Choosing what to commit to and keeping that commitment alive across turnover are different jobs, and training models or people to "want" the right thing is a weak long-horizon control. That part survived.

What I dropped is reading "engineering" as another values process. IEEE 7000 and ISO 26000 look like the first job made into a design workshop. The Common Rule looks like the second. Gelsinger is what is still open: a halt the host cannot overrule on paper (46.112) can still fail when the people who staff it are paid by the same university that wants the trial. If that is "need a better isolated halt," the next artifact is a named stop for a training run. If that is "committees employed by the host are still character," you need isolation from the funder, not another IRB.

two_columnscollapsed

Concrete test, labelled as such. Take one public frontier training announcement — a GPT-class or Claude-class or Gemini-class run with a dated blog post — and score it on two sheets. Do not average them.

Sheet A is IEEE 7000: were relevant values elicited, prioritized, written as ethical value requirements, and traced into the decision to train? A published model card or constitution can fill some of those cells.

Sheet B is the Common Rule halt: is there a named body that can suspend that run, whose disapproval the product organization cannot overrule, with a written reason reported to someone outside the lab?

If A is full and B is empty, elicitation is communications. If B exists and still did not halt under a documented safety miss, Gelsinger is the lab case and you are arguing about isolation from the funder. Either result tells you which object "ethics is engineering" is asking you to build. Mittelstadt's four missing pieces (fiduciary duty, profession, translation method, legal bite) are the checklist for whether sheet B is real or letterhead.