Welcome to the AEQUITAS Framework for AI Fairness

The AEQUITAS Framework is an innovative approach designed to ensure fairness and trust in AI-based decision support systems. It proposes a controlled experimentation environment for developers and users to:

• Assess bias in AI systems by identifying potential sources in data, algorithms, and result interpretation.

• Offer effective methods and engineering guidelines to repair, and mitigate bias when possible.

• Provide fairness-by-design guidelines, methodologies, and software engineering techniques to create new, bias-free systems.

The framework includes an experimentation environment that generates synthetic datasets with various features impacting fairness, allowing laboratory tests.

Real-world use cases in healthcare, human resources, and social challenges are also provided and discussed along with different solutions and comparison.

Note

This project is under active development.

Contents

• Purpose and Scope
  • General Introduction
  • The AEQUITAS project
    • Objectives and Ambitions
    • Methodology
• Glossary
  • AI Stakeholder Identification Methodology (Terms and Definitions)
    • Identification of stakeholders affected by the AI-system (Affectées)
    • Identification of Decisionmakers
    • Identification of Domain Experts and Users
• Framework Components
  • AI fairness in the AI system lifecycle: the holistic AEQUITAS methodology
    • Introduction
    • Fair-by-design Approaches
      • Social, Legal and Ethical Technical Lens
        • Fairness Social Perspective
        • Fairness Legal Perspective
      • Technical Lens
      • Bridging the Gap
    • Information Flow Methodology (IFM): contextualized fairness by bridging social and technical perspectives
      • IFM Model: How the System Is Represented
      • Why the IFM Model Works
      • IFM Methodology: How the Model Is Built and Analysed
      • Summary
    • FairBridge: Converting Legal and Social Principles into AI Fairness Techniques
      • The Meta-Methodology
      • Actors Types
      • Questions/Answers Flow
  • AI Bias Detection
    • Introduction
      • What is Bias Detection?
      • Why is Bias Detection important?
    • Detection Approaches
      • Notation
      • Definitions based on predicted outcome
        • Statistical parity (SP)
        • Conditional Statistical Parity
      • Definitions Based on Predicted and Actual Outcomes
        • Predictive parity
        • False Positive Error Rate Balance
        • False Negative Error Rate Balance
        • Equalized Odds
      • Definitions Based on Predicted Probabilities and Actual Outcome
        • Test-fairness
        • Balance for positive class
        • Balance for negative class
      • Similarity-based measures
        • Causal discrimination
        • Fairness through unawareness
      • Causal Reasoning
        • Counterfactual fairness
    • Bibliography
  • AI Bias Mitigation
    • What is Bias Mitigation?
      • Why is it important?
    • Mitigation Approaches
      • Pre-processing
      • In-Processing
      • Post-Processing
        • References
  • Technology
    • Introduction
    • Architectural Overview
      • Use Case view
        • Users
        • AEQUITAS Primary Use Scenario
        • AEQUITAS Secondary Use Scenarios
• Innovative Techniques for AI Fairness
  • AI Bias Detection & Awareness
  • AI Bias Mitigation
  • AI Fairness: Methodology and Formal Methods
  • Benchmarks and Synthetic Data Generation: different bias polarization
• Use Cases Overview
  • Domain Recruitment
    • Use case HR1: Bias free AI assisted recruiting system
      • Introduction and background
        • Known biases and unfairness
      • Fair-by-Design – Fair Data Collection, Governance and Management methodology
      • Socio-technical analysis using IFM
        • Introduction to IFM
        • IFM Model and Table
        • IFM Analysis
          • Bias Analysis
          • Impact Analysis
        • Suggestions
        • Validation reflections
      • Integration into the experimentation environment
        • Mitigations
        • Results
        • Key Outcomes
      • Use of synthetic data
      • Assessment Alternative Assessment: CV Screening & Keyword Matching
        • Introduction & Problem Statement
        • Pipeline & Methodology
        • Key Findings & Results
        • Limitations & Future Directions
      • Learnings
      • Experimenter Reports
      • Design Process History - A Transparent Approach
    • Use case HR2: Assess and repair job-matching AI-assisted recruiting tool to mitigate gender and other bias
      • Introduction and background
      • Use case description
        • Recruitment Process
        • Known biases and unfairness
      • Akkodis Data and Analysis
        • Data Collection
        • Dataset Structure and Pre-Processing
      • Experimenter Reports
      • Appendix
  • Domain Society and economics
    • Use case S1: AI assisted identification of child abuse and neglect in hospital with implications for socio-economic disadvantaged and racial bias reduction
      • Introduction and background
      • Fair-by-Design methodology: Prohibited Social Scoring Assessment (PSSA)
        • Methodology Overview
        • Key Findings
          • Part 1: System Requirements
          • Part 2: Consequence Requirements
        • Assessment of PSSA
      • Integration into the experimentation environment
        • Experimental phase
          • Manual Testing
          • Automated Testing with LLMs
        • Results and Ethical Validation
        • Key outcomes
      • Learnings and conclusions
      • Appendix
    • Use case S2: Unfair Inequality in Education
  • Domain Healthcare
    • Use case HC1: AI assisted identification of dermatological disease for diversity and inclusion in dermatology
      • Introduction and background
      • Fair-by-Design methodology
        • Methodology Overview
        • Key Findings
        • Assessment of FMM
      • Socio-technical analysis using IFM
        • Introduction to Socio-Technical Context: Dermatological case
        • Scenario 1: Child-Led & Guardians AI Use
          • Follow-Up IFM for True Positives (TP) and False Positives (FP)
          • Key takeaway
          • Implications for False Negatives (FN)
        • Scenario 2: AI Used by the Family Doctor
          • Suggestions
        • Validation Reflections
      • Integration into the experimentation environment
        • Results and Analysis
        • Links to FbD Methodology and IFM
        • Key Outcomes
      • Use of syntetic data
        • Interpretation and Methodological Links
        • Boundaries and Limitations
      • Learnings and conclusions
      • Experimenter Reports
      • Appendix
      • Design Process History - A Transparent Approach
    • Use case HC2: Bias-aware prediction of ECG healthcare outcomes
      • Introduction and background
      • Bias in ECG
      • Synthetic ECG
      • Fair-by-Design – Fair Output Interpretation Methodology
        • Methodology Overview
        • Key Findings
          • Purpose identification
          • Data Source & Representation
          • Outcome evaluation
        • Assessment of FOIM
      • Integration into the experimentation environment
        • Method & data
          • Data processing
          • Overview of dataset
        • Analysis using the AEQUITAS Experimenter
        • Bias repair using synthetic data
      • Learnings and conclusion
      • Appendix
  • Discussion
    • Methodological Coherence
    • Cross-Domain Insights
    • Technical and Ethical Learnings
    • Validation of AEQUITAS as a Pre-Regulatory Instrument
    • Overall Impact
  • Conclusions
• Pills & Tutorials
  • AI Ethics
  • AI Fairness
  • AEQUITAS Framework
  • Interactive Case Studies
  • Guidelines and Resources
• START EXPERIMENTING
  • Fair-by-Design Methodology
  • Experimentation Environment