Computer Ethics & Society
Computing shapes society at every level — understanding your ethical obligations as an engineer is not optional, it is the difference between building tools that liberate and ones that harm.
Overview
Every line of code you write has the potential to affect millions of people. The software that runs social networks, medical devices, voting systems, and criminal-risk scoring tools does not exist in a moral vacuum. Computer ethics is the rigorous study of those effects and the professional obligations they create.
This module treats ethics not as a list of rules to memorize but as an engineering discipline: you will learn frameworks that help you reason carefully under uncertainty, evaluate tradeoffs, and defend your decisions. Just as you would not ship a bridge without a stress analysis, you should not ship a system that touches people without a careful analysis of its potential harms and benefits.
We cover the full arc: the philosophical foundations that anchor every major framework; the specific intellectual-property regime that governs software (copyright, patents, trade secrets, and the radical alternative represented by free and open-source software); the structural inequity called the digital divide; the formal ethics codes of ACM and IEEE-CS; and the hard, contested problems at the frontier — algorithmic bias, surveillance capitalism, AI ethics, and the responsibilities of whistleblowers.
By the end of this module you will be able to reason rigorously about a real-world case, identify which stakeholders are affected, apply multiple ethical frameworks to the same situation, locate the relevant legal and professional constraints, and articulate a defensible position. That is the same skill expected of a senior engineer or engineering manager at any responsible organization.
Software engineers are among the most leveraged professionals in human history. A single engineer at a major platform company may write code that runs on a billion devices. The asymmetry between the effort of writing code and the scale of its consequences means that the ethical mistakes of engineers can propagate faster and wider than those of almost any other profession. The 2016 Cambridge Analytica scandal, the deployment of biased recidivism-prediction algorithms in US courts (COMPAS), the use of facial recognition by authoritarian governments, and the 2010 Flash Crash caused partly by automated trading all trace directly to engineering decisions made by individuals who either did not consider the ethical dimensions of their work or did not have the tools to reason about them.
In real software development, ethical issues are not neatly labeled. They show up as product decisions ("should we make the dark pattern slightly darker?"), infrastructure decisions ("should we log this user-behavior data?"), and architectural decisions ("should we build a feature that is technically feasible but whose misuse is predictable?"). Professional ethics codes — the ACM Code of Ethics, the IEEE-CS Software Engineering Code — are not merely aspirational documents. They are the profession's collective answer to "what do we owe to the people affected by our work?" Understanding them, and the philosophical tradition behind them, gives you a principled vocabulary for the hardest conversations you will have in your career.
Ethical Decision-Making Framework for Engineers
A structured process for ethical reasoning in engineering contexts: begin with stakeholder analysis, apply multiple philosophical frameworks, check for convergence, consult professional codes, and escalate when necessary. Note that this is a reasoning scaffold, not an algorithm — the quality of the analysis depends on honest engagement with each step.
The concepts
01Ethical Frameworks: The Philosopher's Toolkit
Before applying ethics to computing, you need the underlying frameworks that philosophers have refined over centuries. There are three major families.
Consequentialism judges actions by their outcomes. The most influential version is utilitarianism (Bentham, Mill): an action is right if it produces the greatest good for the greatest number. Quantified: maximize ∑ utility over all affected parties. Consequentialism is natural for engineers who think in optimization — but it has well-known failure modes. It can justify harming a minority to benefit a majority (the "utility monster" problem), and it requires you to predict consequences you often cannot know in advance.
Deontology (Kant) judges actions by whether they conform to universal duties, regardless of outcome. Kant's Categorical Imperative has two forms you must know: (1) Universalizability — act only according to a maxim you could will to be a universal law. (2) Humanity formulation — always treat persons as ends in themselves, never merely as means. Applied to computing: a surveillance system that treats users as data-producing objects rather than autonomous persons fails the humanity formulation even if it produces aggregate social benefit.
Virtue Ethics (Aristotle) asks not "what rule applies?" but "what would a person of good character do?" The goal is eudaimonia (flourishing) achieved by cultivating virtues: honesty, courage, justice, prudence. Professional ethics codes often embed virtue ethics: the ACM code asks you to be honest and trustworthy — not just to follow rules, but to become a certain kind of person.
Contractarianism (Rawls) grounds ethics in what rational agents would agree to from behind a "veil of ignorance" — if you did not know whether you would be a privileged or marginalized user of a system, what rules would you choose? This framework is particularly powerful for thinking about the digital divide and algorithmic fairness: a surveillance system that disproportionately harms the poor would not be chosen by anyone who did not know in advance that they were rich.
In practice, ethical reasoning uses all four frameworks as lenses. Convergence across frameworks strengthens a conclusion; divergence signals genuine moral complexity that requires more careful analysis.
# A minimal consequentialist decision model (pseudocode)
# Demonstrates why pure utility maximization is insufficient
def evaluate_action(action, stakeholders):
"""
Returns total utility of action across all stakeholders.
Problem: this can justify harm to minority if majority gains enough.
"""
total = 0
for person in stakeholders:
delta = estimate_utility_change(person, action)
total += delta
return total
# Rawlsian improvement: maximin criterion
def rawls_evaluate(action, stakeholders):
"""
Maximize the minimum utility — protect the worst-off.
"""
utilities = [estimate_utility_change(p, action) for p in stakeholders]
return min(utilities) # choose action that maximizes this
02Intellectual Property: Copyright, Patents, and Trade Secrets
Intellectual property (IP) law defines who owns software and ideas. As an engineer you will encounter all three major regimes constantly.
Copyright attaches automatically to any original creative work fixed in a tangible medium — including source code. You do not need to register. In the US, copyright lasts the author's life plus 70 years (for individuals) or 95–120 years for works-for-hire. Copyright protects expression, not ideas. You can copyright the specific implementation of a sorting algorithm, but not the concept of sorting. This idea/expression dichotomy is why independent reimplementation is legal: if you write a fresh implementation of quicksort from scratch without copying another's code, you do not infringe. The APIs-as-copyrightable question was definitively answered in Google v. Oracle (2021): the Supreme Court held that Google's reimplementation of Java APIs in Android was fair use, preserving the right to reimplement interface specifications.
Patents protect inventions — novel, non-obvious, useful processes or machines — for 20 years from filing. Software patents are controversial because (a) the patent office long lacked expertise to assess prior art in software, leading to thousands of overbroad patents, and (b) software ideas often build incrementally and cross-licensing costs can impede innovation. The Supreme Court's Alice Corp. v. CLS Bank (2014) decision significantly narrowed patentable software subject matter by invalidating patents on abstract ideas implemented on a computer.
Trade secrets protect confidential business information (source code, algorithms, training data, customer lists) indefinitely, as long as the owner takes reasonable steps to maintain secrecy. Unlike patents, there is no disclosure requirement and no expiration — but protection ends if the secret leaks or is independently discovered. The Defend Trade Secrets Act (2016) federalized trade secret law in the US.
DMCA (Digital Millennium Copyright Act, 1998): Section 1201 prohibits circumvention of technological protection measures (TPMs), even for lawful purposes. This has profound implications for security research: reverse-engineering a device to find vulnerabilities can be a DMCA violation. Section 512 provides "safe harbor" for platforms that remove infringing content on notice — the legal foundation for every content moderation system.
The key engineering takeaway: when you write code at work, your employer almost certainly owns it (work-for-hire doctrine). When you use open-source libraries, the license terms are legally binding. When you find a vulnerability, the DMCA and computer fraud statutes constrain how you disclose it.
03Free and Open-Source Software: Philosophy and Licensing
The free software movement, launched by Richard Stallman in 1983 with the GNU Project, is one of the most consequential political-philosophical movements in computing. Understanding it requires distinguishing free software (a values-based movement) from open-source software (a development methodology).
Free software (FSF definition) means software that respects four essential freedoms: (0) to run the program for any purpose; (1) to study and change the source code; (2) to redistribute copies; (3) to distribute modified versions. "Free" means freedom (libre), not price (gratis).
The GPL and copyleft: The GNU General Public License (GPL) is the legal mechanism that enforces these freedoms. Its key innovation is copyleft: if you distribute a modified GPL program, you must release the source under the same terms. Copyleft uses copyright law to guarantee freedom — it is a "viral" license in the sense that GPL code cannot be absorbed into proprietary software without triggering the disclosure obligation. The LGPL weakens this for libraries (linking does not trigger copyleft); the AGPL extends it to network services (running a modified GPL program as a network service triggers source disclosure).
Permissive licenses (MIT, BSD, Apache 2.0) allow proprietary redistribution with minimal requirements (typically just attribution). They maximize adoption but sacrifice the guarantee that improvements remain free. The tension between copyleft and permissive licenses is a genuine values debate: permissive licenses grow the total ecosystem faster; copyleft licenses grow the free ecosystem and prevent "free-riding" by corporations.
The open-source movement (ESR, Torvalds, 1998) reframed free software in pragmatic terms: open source produces better software because of many-eyeballs review, rapid iteration, and developer motivation. The OSI certifies licenses as "open source." All free software licenses are open source; not all open source licenses are copyleft.
Creative Commons extends similar licensing concepts to non-software works: documentation, datasets, educational material. Relevant for ML researchers: many datasets carry CC licenses whose terms restrict commercial use (CC-BY-NC) or require share-alike (CC-SA).
Key case: Jacobsen v. Katzer (Federal Circuit, 2008) established that open-source license terms are conditions (not merely covenants), meaning violation is copyright infringement (injunctive relief available), not just breach of contract (money damages only). This gave OSS licenses genuine legal teeth.
# Minimal illustration of license compatibility logic
# (This is the kind of analysis a build system must do)
LICENSE_COMPATIBILITY = {
# (dependency_license, your_project_license) -> compatible?
('MIT', 'MIT'): True,
('MIT', 'GPL3'): True, # MIT can be used in GPL3 project
('GPL3', 'MIT'): False, # GPL3 CANNOT be used in MIT project
('GPL3', 'GPL3'): True,
('LGPL3', 'MIT'): False, # LGPL3 dynamic-link exception doesn't apply
('Apache2','GPL3'): True, # Apache2 compatible with GPL3 (not GPL2)
('Apache2','GPL2'): False, # Apache2 NOT compatible with GPL2
}
def can_use(dep_license, project_license):
return LICENSE_COMPATIBILITY.get((dep_license, project_license), False)
# Lesson: GPL3 is copyleft — if your project is proprietary (e.g. 'Proprietary'),
# you CANNOT include a GPL3 dependency.
print(can_use('GPL3', 'Proprietary')) # False
04The Digital Divide: Structural Inequality in Access
The digital divide refers to the gap between those who have meaningful access to digital technologies and those who do not. It operates at multiple levels:
Level 1 — Access: Do you have a device and connectivity? Globally, as of the mid-2020s, roughly 2.6 billion people remain offline. Within wealthy countries, the divide tracks income, age, disability, and geography (urban vs. rural). The COVID-19 pandemic made this viscerally concrete: students without home internet could not attend school; workers without devices could not work remotely.
Level 2 — Skills: Even people with devices may lack the literacy to use them effectively — to evaluate the credibility of online information, protect their privacy, or use productivity tools. This is sometimes called the "second-level digital divide."
Level 3 — Meaningful use: Even skilled users may be locked out of economic opportunity if platform algorithms, language barriers, or accessibility failures exclude them. A screen reader that cannot parse a JavaScript-heavy SPA, or a loan-approval algorithm trained on historically discriminatory data, are examples of meaningful-use failures.
Policy responses: The US FCC's E-Rate program subsidizes broadband for schools and libraries. The FCC's Universal Service Fund supports rural carriers. The EU's Digital Compass plan targets 100% broadband coverage. International initiatives include UN Sustainable Development Goal 9c (universal internet access by 2030).
Engineering responsibility: The digital divide is not just a policy problem. It is an engineering problem. Decisions about minimum supported device specs, required bandwidth, accessibility standards (WCAG 2.1), default language, and data costs determine who can use your software. "Ship fast on the latest iPhone on fiber" implicitly excludes billions of people. Progressive enhancement — building a functional baseline that works on slow connections and old devices, then layering richer experiences on top — is the engineering discipline that operationalizes inclusion.
The double-edged sword: Digital technologies can reduce inequality (mobile banking reaching the unbanked via M-Pesa in Kenya) or amplify it (surveillance capitalism extracting data from populations with little bargaining power). The direction depends on design choices made by engineers.
/* Progressive enhancement: a real accessibility pattern
Build the baseline first; JavaScript is an enhancement,
not a prerequisite. This is how you avoid excluding
users on slow connections or with JS disabled. */
<!-- Baseline: works with no JS, works on any browser -->
<form action="/search" method="GET">
<input name="q" type="search" aria-label="Search" />
<button type="submit">Search</button>
</form>
<script>
// Enhancement: upgrade to instant search if JS available
const input = document.querySelector('[name=q]');
input?.addEventListener('input', debounce(async (e) => {
const res = await fetch(`/api/search?q=${e.target.value}`);
renderResults(await res.json());
}, 300));
</script>
05Professional Ethics Codes: ACM and IEEE-CS
Professional ethics codes are the profession's collective statement of its obligations to society. The two most important for software engineers are the ACM Code of Ethics and Professional Conduct (updated 2018) and the ACM/IEEE-CS Software Engineering Code of Ethics (1999).
ACM Code of Ethics (2018) is organized around four sections:
- General ethical principles: contribute to society and human well-being; avoid harm; be honest and trustworthy; be fair and take action not to discriminate; respect intellectual property; respect privacy; honor confidentiality.
- Professional responsibilities: achieve and maintain competence; give and accept professional review; provide comprehensive and honest evaluations; maintain systems and processes to protect privacy and security.
- Professional leadership principles: for those in leadership roles, ensure policies and processes enable ethical practice.
- Compliance: uphold and promote the code.
The 2018 revision added explicit language about algorithmic discrimination, privacy, and the responsibility to consider downstream effects — a direct response to the AI ethics crises of the 2010s.
SE Code (ACM/IEEE-CS, 1999) has eight principles, centered on the practitioner acting consistently with the public interest: PUBLIC, CLIENT AND EMPLOYER, PRODUCT, JUDGMENT, MANAGEMENT, PROFESSION, COLLEAGUES, SELF. The mnemonic is PC4M + PS or just the first letters. The most important are PUBLIC (public interest above all) and PRODUCT (best possible product given constraints).
How codes function in practice: Ethics codes serve four functions: (1) Aspirational — setting the bar you aim for. (2) Educational — giving practitioners a shared vocabulary. (3) Disciplinary — professional bodies can revoke membership for violations. (4) Legal cover — in litigation, conformance to a professional code is evidence of due care.
Limitations: Professional ethics codes are not enforceable the way law is. Unlike medicine or law, software engineering does not have universally mandatory licensure. The IEEE Software Engineering Professional (CSEP) and Canada's P.Eng are exceptions. The lack of universal licensure means individual engineers have less structural protection when they raise ethical concerns — which makes the culture of ethics in organizations more important.
Whistleblowing: When internal channels fail, the ACM code explicitly supports raising concerns through external channels. Famous cases: Frances Haugen (Facebook), Edward Snowden (NSA), Erin Brockovich (analog, but the pattern is identical). The tension is between loyalty to employer and obligation to public — the SE Code resolves this clearly: public interest comes first.
06Algorithmic Bias and Fairness
Algorithmic bias occurs when a computational system produces systematically unfair outcomes for identifiable groups. It is one of the most technically rich and socially consequential problems in applied computer science.
Sources of bias:
- Historical bias: training data reflects past discrimination. A hiring model trained on historical decisions will encode historical bias against women or minorities.
- Representation bias: training data underrepresents some groups. Early face recognition systems had dramatically higher error rates on darker skin tones because training sets were majority-white.
- Measurement bias: the proxy variable being predicted is not the true quantity of interest. COMPAS predicts "recidivism" using zip code, which correlates with race — a biased proxy.
- Aggregation bias: a single model trained on a heterogeneous population fits no subgroup well.
- Deployment bias: a model used for a purpose different from its validation context.
Fairness metrics — the impossibility theorem: Chouldechova (2017) and Kleinberg et al. (2016) proved that several intuitive fairness criteria are mathematically incompatible when base rates differ across groups. Specifically, you cannot simultaneously achieve:
- Calibration: the score means the same thing for all groups (P(Y=1 | score=s, group=g) is the same for all g).
- Equal false positive rates across groups.
- Equal false negative rates across groups.
This is not a software bug. It is a theorem. It means that "fair" is not a single well-defined objective — you must choose which notion of fairness to optimize for, and that choice is a values decision, not a technical one.
Key fairness definitions:
- Individual fairness: similar individuals should receive similar outcomes.
- Group fairness / demographic parity: positive outcome rates should be equal across groups.
- Equalized odds: both TPR and FPR should be equal across groups.
- Counterfactual fairness: the outcome should be the same as in a counterfactual world where only the protected attribute changed.
Mitigation techniques operate at three stages: pre-processing (rebalance or reweight training data), in-processing (add fairness constraints to the learning objective), post-processing (threshold calibration per group). Each has tradeoffs in accuracy, fairness type achieved, and interpretability.
Case study — COMPAS: ProPublica's 2016 analysis of the COMPAS recidivism-prediction tool found that Black defendants were nearly twice as likely as white defendants to be falsely flagged as future criminals. Northpointe (the vendor) responded that the tool was calibrated — scores meant the same thing across races. Both were correct. This is the impossibility theorem in action.
import numpy as np
# Toy illustration: measuring demographic parity and equalized odds
# y_true: ground truth, y_pred: model predictions, group: 0 or 1
def fairness_audit(y_true, y_pred, group):
results = {}
for g in [0, 1]:
mask = (group == g)
yt, yp = y_true[mask], y_pred[mask]
tp = np.sum((yt == 1) & (yp == 1))
fp = np.sum((yt == 0) & (yp == 1))
fn = np.sum((yt == 1) & (yp == 0))
tn = np.sum((yt == 0) & (yp == 0))
tpr = tp / (tp + fn) if (tp + fn) > 0 else 0 # sensitivity
fpr = fp / (fp + tn) if (fp + tn) > 0 else 0 # false alarm rate
pos_rate = np.mean(yp) # demographic parity numerator
results[g] = {'tpr': tpr, 'fpr': fpr, 'pos_rate': pos_rate}
print(f"Demographic parity gap: {abs(results[0]['pos_rate'] - results[1]['pos_rate']):.3f}")
print(f"TPR gap (equalized odds): {abs(results[0]['tpr'] - results[1]['tpr']):.3f}")
print(f"FPR gap (equalized odds): {abs(results[0]['fpr'] - results[1]['fpr']):.3f}")
# Key: you cannot make all three gaps simultaneously zero when base rates differ
return results
07Privacy: Technical Foundations and Legal Frameworks
Privacy is not merely a legal concept — it has a precise technical meaning and a rich philosophical tradition.
Philosophical foundations: Warren and Brandeis (1890) defined privacy as "the right to be let alone." Westin (1967) described it as "the claim of individuals to determine for themselves when, how, and to what extent information about them is communicated to others." Nissenbaum (2004) proposed contextual integrity: privacy is violated when information flows in ways that violate the norms of the context in which it was shared. Medical information shared with a doctor flows appropriately to another treating physician; it does not flow appropriately to an employer.
Legal frameworks:
- GDPR (EU, 2018): Rights to access, rectification, erasure ("right to be forgotten"), portability, and objection to automated decision-making. Consent must be freely given, specific, informed, and unambiguous. Penalties up to 4% of global annual revenue.
- CCPA/CPRA (California): Similar rights in the US. "Do Not Sell My Personal Information" opt-out requirement.
- HIPAA (US, 1996): Health information. FERPA: Educational records. COPPA: Children under 13.
- Privacy by Design (Cavoukian): privacy should be built into systems as a default, not bolted on afterward. This is now a regulatory expectation under GDPR Art. 25.
Technical mechanisms:
- Differential privacy (Dwork, 2006): a rigorous mathematical definition. A randomized algorithm M is ε-differentially private if for any two datasets D, D' differing in one row, and any output S: P[M(D) ∈ S] ≤ e^ε · P[M(D') ∈ S]. Small ε means strong privacy. Used by Apple (WWDC 2016) and US Census Bureau (2020 decennial census) to release aggregate statistics without exposing individuals.
- k-anonymity: every record in a released dataset is indistinguishable from at least k-1 other records on quasi-identifiers. Narayanan and Shmatikoff (2008) showed Netflix's "anonymized" dataset could be re-identified using IMDb as an external reference — a landmark result on the limits of k-anonymity.
- Homomorphic encryption: compute on encrypted data without decrypting. Fully homomorphic encryption (Gentry, 2009) is theoretically possible but computationally expensive; partially homomorphic schemes (Paillier, ElGamal) are practical for specific operations.
- Federated learning: train ML models across devices without centralizing raw data. Gradient updates are shared, not raw examples. Still vulnerable to membership inference and gradient inversion attacks.
The key engineering principle: data minimization. Collect only what you need, retain only as long as necessary, encrypt in transit and at rest, and assume every dataset you hold will eventually be breached.
import numpy as np
# Differential privacy: adding calibrated Laplace noise to a query
# Query: "How many users in dataset satisfy predicate P?"
# Sensitivity = 1 (adding/removing one user changes count by at most 1)
def dp_count(true_count: int, epsilon: float, sensitivity: float = 1.0) -> float:
"""
Returns a differentially private estimate of true_count.
epsilon: privacy budget. Smaller = more private, less accurate.
Laplace mechanism: add noise ~ Laplace(0, sensitivity/epsilon)
"""
scale = sensitivity / epsilon
noise = np.random.laplace(0, scale)
return true_count + noise
# Demonstration
true = 1000
for eps in [0.1, 1.0, 10.0]:
estimates = [dp_count(true, eps) for _ in range(5)]
print(f"epsilon={eps:.1f}: {[f'{e:.0f}' for e in estimates]}")
# epsilon=0.1 -> high noise (strong privacy), epsilon=10.0 -> low noise (weak privacy)
08Surveillance Capitalism and Platform Ethics
Surveillance capitalism is Shoshana Zuboff's term (from her 2019 book) for the economic logic underlying most major internet platforms: human behavioral data is extracted as a raw material, refined into predictive products, and sold to advertisers. The key insight is that the product is not you — it is a behavioral prediction product sold to others. You are the raw material.
The attention economy: platforms maximize engagement (time-on-site) because more engagement generates more behavioral data. This creates systematic pressure toward outrage, anxiety, and addiction, because these emotional states are more engaging than calm satisfaction. The "social dilemma" is not a conspiracy — it is the emergent result of gradient descent on an engagement objective.
Dark patterns: UX designs that nudge users against their own interests. Categories defined by Gray et al. (2018): trick questions, sneak into basket, roach motel (easy to get in, hard to get out), privacy zuckering (deceiving into sharing more than intended), misdirection, hidden costs, disguised ads, forced continuity, friend spam. The FTC has brought enforcement actions against dark patterns (Amazon's difficult Prime cancellation flow, 2023).
Network effects and lock-in: platforms with strong network effects (social networks, messaging apps) create switching costs that allow them to maintain practices users dislike. This is the structural source of platform power: the value of the network is proportional to its size, so the dominant platform can be worse than a competitor and still retain users.
Regulatory responses: GDPR's consent requirements directly target behavioral advertising. The EU Digital Markets Act (2022) designates "gatekeepers" with special obligations including interoperability and data portability — directly attacking lock-in. US legislative proposals (AICOA, KOSA) address algorithmic amplification of harmful content to minors.
The engineer's position: Many engineers at platform companies did not intend to build addictive systems. The ethical question is: when you understand what the engagement objective does to human psychology, do you have an obligation to raise it? The ACM code says yes. The practical question is how — internal advocacy, product design alternatives, and in extreme cases, whistleblowing.
Responsible design alternatives: Time well spent (Tristan Harris, Center for Humane Technology), meaningful engagement metrics (did the user accomplish their goal?), friction by design (confirm-before-scroll, daily usage summaries), and algorithmic transparency (explainable feeds) are all engineering-level interventions.
09AI Ethics: Emerging Principles and Governance
The deployment of large-scale AI systems has created a new class of ethical problems that existing frameworks were not designed to handle. The field of AI ethics has converged on a set of principles (with significant overlap across major frameworks from IEEE, EU, Google, Microsoft, the White House EO, and academia).
Core principles:
- Transparency: AI systems should be explainable. Stakeholders should be able to understand why a decision was made. This is both an ethical principle (autonomy requires understandable treatment) and a legal requirement under GDPR Art. 22 (right to explanation for automated decisions).
- Accountability: there must be a human responsible for the system's outcomes. The "responsibility gap" — where no individual or organization is clearly liable for an AI system's harm — is a governance failure.
- Fairness: as discussed in the algorithmic bias section. AI systems should not discriminate on protected characteristics.
- Safety: AI systems should be robust to adversarial inputs, distribution shift, and failure modes. Autonomous systems should fail safe.
- Privacy: covered above.
- Human oversight: especially for high-stakes decisions (medical, criminal, financial), there should be a meaningful human in the loop — not a rubber-stamp review of thousands of decisions per day.
Regulatory landscape:
- EU AI Act (2024): risk-based framework. Prohibited uses (social scoring, real-time biometric surveillance in public spaces), high-risk uses (CV screening, credit scoring, medical devices, critical infrastructure) with mandatory conformity assessment, limited-risk uses (chatbots) with transparency obligations.
- US Executive Order on Safe, Secure, and Trustworthy AI (2023): NIST AI RMF, mandatory reporting for frontier models, watermarking for AI-generated content.
- NIST AI Risk Management Framework (2023): voluntary framework with four functions: GOVERN, MAP, MEASURE, MANAGE.
Explainability techniques: LIME (Local Interpretable Model-agnostic Explanations) fits a simple surrogate model locally around a prediction. SHAP (SHapley Additive exPlanations) uses Shapley values from cooperative game theory to assign feature attributions. Attention visualization for transformers. Counterfactual explanations: "the loan was denied; if your income were $5,000 higher, it would have been approved."
The value alignment problem: for sufficiently capable AI systems, the hardest problem is ensuring the system pursues human values, not a proxy that diverges from those values at scale (Goodhart's Law: when a measure becomes a target, it ceases to be a good measure). This is the theoretical foundation for AI safety research.
# SHAP explanation sketch (conceptual — real use requires shap library)
# Shapley value: average marginal contribution of feature i
# across all possible orderings of features.
# For a model f and input x:
# phi_i = sum over all subsets S not containing i of:
# (|S|! * (n - |S| - 1)! / n!) * [f(S + {i}) - f(S)]
# In practice:
import shap # pip install shap
# explainer = shap.TreeExplainer(trained_model)
# shap_values = explainer.shap_values(X_test)
# shap.summary_plot(shap_values, X_test)
# Interpretation: shap_values[i][j] is the contribution of
# feature j to the prediction for sample i, relative to the
# base rate. Positive = pushed prediction higher.
# This lets you say: "income contributed +0.3 to your credit score,
# but zip code contributed -0.5" — auditable, actionable.
10The Social Responsibility of Computing: Case Studies
Abstract principles become concrete through cases. The following canonical cases illustrate the full range of social responsibility issues.
Therac-25 (1985-1987): A radiation therapy machine killed or seriously injured at least six patients due to race conditions in the control software. The race condition was only triggerable by experienced operators who typed fast. The software had removed hardware safety interlocks from the previous model, assuming software was more reliable. Key lessons: software safety is a system property, not a code property; regression testing must cover the full human-machine interface; removing hardware safeguards for software replacements requires extraordinary evidence.
Boeing 737 MAX MCAS (2018-2019): 346 people died in two crashes. The Maneuvering Characteristics Augmentation System (MCAS) was added to compensate for aerodynamic changes from engine repositioning. Pilots were not adequately informed of its existence. A single faulty angle-of-attack sensor could trigger it repeatedly, overriding pilot inputs. Key lessons: safety-critical systems should be fail-safe (not fail-unsafe); single points of failure are unacceptable; documentation and training are part of the system.
Facebook and the 2016 election: Cambridge Analytica harvested data on 87 million Facebook users via an app that exploited the Graph API's permission to pull friends' data. The data was used for psychographic profiling and targeted political advertising. Key lessons: API design choices have political consequences; "available to developers" is not consent; third-party data access requires ongoing auditing.
COMPAS and criminal justice (covered in algorithmic bias section).
Zoom's end-to-end encryption claim (2020): Zoom marketed its product as having end-to-end encryption when in fact calls were encrypted only between endpoints and Zoom's servers (transport encryption), allowing Zoom to access call content. This is a precise technical distinction with major privacy and legal implications. Key lesson: precision in security claims is an ethical obligation.
How to analyze a case: Use the stakeholder analysis method. (1) Identify all stakeholders (users, non-users affected, developers, employer, profession, society). (2) Identify harms and benefits for each. (3) Apply each ethical framework. (4) Identify applicable legal and professional obligations. (5) Identify what a person of good character would have done at each decision point. (6) What systemic changes would prevent recurrence?
Key insights
- ✦Ethical reasoning is not a single algorithm — it is the skill of applying multiple frameworks (consequentialism, deontology, virtue ethics, contractarianism) and reasoning carefully when they diverge.
- ✦The impossibility theorem (Chouldechova, Kleinberg) proves that common fairness criteria are mathematically incompatible when base rates differ across groups — 'fair' is a values choice, not a technical default.
- ✦Copyleft is a legal judo move: it uses copyright law to guarantee the freedoms that copyright normally restricts. The GPL does not abolish IP law — it weaponizes it for freedom.
- ✦Differential privacy is a mathematical guarantee, not a heuristic: the privacy budget epsilon quantifies exactly how much information is leaked about any individual, allowing principled privacy-utility tradeoffs.
- ✦The digital divide is an engineering problem: decisions about minimum device specs, required bandwidth, progressive enhancement, and accessibility standards determine who can participate in digital society.
- ✦Professional ethics codes (ACM, IEEE-CS) are not optional guidelines — they create due-care obligations, inform legal standards of professional conduct, and explicitly place public interest above employer interests.
- ✦Platform ethics is structurally driven: dark patterns, addictive design, and surveillance capitalism emerge from engagement objectives optimized by gradient descent, not from individual bad actors.
- ✦The Therac-25 and Boeing 737 MAX cases show that removing hardware safety checks for software equivalents requires extraordinary evidence — software introduces new failure modes (race conditions, sensor fusion errors) that hardware interlocks do not have.
Common pitfalls
- ⚠Treating 'ethical' as binary: real engineering decisions involve tradeoffs between legitimate competing values, not clean good-vs-evil choices. The impossibility theorem is the formal proof of this.
- ⚠Conflating legality and ethics: something can be legal and deeply unethical (historical slavery, current surveillance capitalism), or illegal and ethical (civil disobedience, some whistleblowing). Legal compliance is a floor, not a ceiling.
- ⚠Ignoring downstream effects and non-users: stakeholder analysis that only considers paying customers misses the populations most harmed by many systems (e.g., third parties profiled by behavioral advertising).
- ⚠Assuming 'anonymized data' is safe: Narayanan and Shmatikoff's Netflix re-identification and Sweeney's medical record re-identification demonstrate that k-anonymity is not sufficient against auxiliary information attacks.
- ⚠Treating the ACM/IEEE codes as abstract aspirations rather than actionable obligations with real professional and legal consequences.
- ⚠Confusing open-source with public domain: open-source code is still copyrighted and its license terms are legally binding conditions, not mere guidelines. License incompatibility is a real legal exposure.
- ⚠Assuming a technically accurate system is a fair one: a well-calibrated model can still be discriminatory in application, and a model with equal accuracy across groups can still have vastly different error rates on subgroups.
- ⚠Believing that ethical concerns are someone else's job: at most software companies there is no ethics officer reviewing your code. The engineer is the last line of defense.
Practice
- 01medium
COMPAS fairness audit. Download ProPublica's COMPAS dataset (publicly available on GitHub). Implement the three fairness metrics discussed in the module: demographic parity gap, equalized odds (TPR gap and FPR gap), and calibration. Reproduce ProPublica's finding. Then implement Northpointe's counter-argument (calibration parity). Write a 2-paragraph explanation of why both analyses are correct simultaneously, citing the impossibility theorem.
- 02medium
License compatibility checker. Build a Python tool that, given a software project's dependency tree (as a JSON file listing each package and its SPDX license identifier), determines whether the project can be released under a given target license. Handle the GPL-2.0, GPL-3.0, LGPL-2.1, LGPL-3.0, Apache-2.0, MIT, BSD-2-Clause, and BSD-3-Clause licenses. Your compatibility matrix should be grounded in the actual OSI and FSF guidance.
- 03medium
Differential privacy implementation. Implement a differentially private histogram over a synthetic dataset of ages (integers 0-100). Use the Laplace mechanism. Write a function that takes a dataset, a privacy budget epsilon, and produces a DP histogram. Then plot the true histogram vs. the DP histogram for epsilon in {0.1, 0.5, 1.0, 10.0}. For each epsilon, compute the L1 error between true and DP histograms over 100 trials. Explain the privacy-utility tradeoff you observe.
- 04hard
Ethical case analysis. Analyze the Facebook Emotional Contagion experiment (Kramer et al., 2014). In that experiment, Facebook manipulated the emotional content of 689,003 users' news feeds without their explicit consent to study emotional contagion. Using the four-framework method (consequentialist, deontological, virtue, Rawlsian) and stakeholder analysis, write a structured 1500-word ethical assessment. Identify which ACM Code of Ethics principles were potentially violated. Propose what a responsible research design would have looked like.
- 05easy
Progressive enhancement audit. Pick any major website you use. Test it under three conditions: (a) normal conditions, (b) JavaScript disabled (use browser dev tools), (c) throttled to 3G (use Chrome's network throttle). Document what functionality is lost, what accessibility features break, and what data is consumed per page load. Write a technical report identifying three specific changes the engineering team could make to improve inclusivity, citing WCAG 2.1 criteria and the digital divide framework.
- 06hard
Free software philosophy debate. Write a structured argument for BOTH sides of the following proposition: 'The GPL is harmful to software freedom because its copyleft restrictions prevent free software from being used in the most beneficial contexts, including humanitarian and educational software that must be combined with proprietary components.' Use Stallman's four freedoms framework, the pragmatic open-source argument (ESR), and at least one concrete real-world licensing conflict as evidence.
Resources
The primary professional ethics document for computer scientists. Read the full text, not just the principles — the elaborations are where the actionable guidance lives.
The eight-principle code specifically for software engineers. More detailed and prescriptive than the general ACM code.
The definitive scholarly account of surveillance capitalism. Dense but essential. At minimum read the introduction and chapters 1-3.
Accessible, concrete account of how algorithms embed and amplify bias in high-stakes domains: recidivism, credit, hiring, education.
The philosophical foundation of the free software movement, in Stallman's own words. Read 'The GNU Project' and 'Why Software Should Be Free.'
The original ProPublica investigation that sparked the algorithmic fairness debate. Primary source material for the COMPAS case study.
Excellent accessible introduction to differential privacy from the Harvard Privacy Tools project.
MIT OCW. The ethics and fairness modules are directly applicable to this topic.
The definitive treatment of contextual integrity as a framework for privacy. Essential for understanding why 'publicly available data' is not the same as 'data people consented to share.'
Unofficial but well-maintained portal for the EU AI Act with article-by-article navigation. Essential for understanding the emerging regulatory landscape.
The US government's voluntary framework for AI risk management. Increasingly referenced in procurement and legal contexts.
The canonical reference for OSI-approved licenses with compatibility guidance. Use this to verify license claims.
Checkpoints — you can now…
- ✓You can apply all four ethical frameworks (consequentialist, deontological, virtue ethics, Rawlsian) to a concrete engineering scenario and identify where they converge and where they diverge.
- ✓You can explain the idea/expression dichotomy in copyright, the difference between GPL and permissive licenses, and why license compatibility is a legal (not just social) obligation.
- ✓You can define differential privacy mathematically (the epsilon definition), implement the Laplace mechanism, and explain the privacy-utility tradeoff.
- ✓You can state and apply the Chouldechova-Kleinberg impossibility theorem: you can demonstrate with a concrete numeric example why calibration and equalized odds cannot simultaneously hold when base rates differ.
- ✓You can perform a complete stakeholder analysis on a real case (COMPAS, Therac-25, or an equivalent), identify applicable ACM Code of Ethics principles, and write a defensible ethical assessment.
- ✓You can explain the three levels of the digital divide and identify at least three concrete engineering decisions that improve digital inclusivity.
- ✓You can describe the key provisions of GDPR, the EU AI Act risk tiers, and the NIST AI RMF, and explain how each applies to a hypothetical ML system you might build.