Northwestern Panel, March 10 2022

AI Regulation: Will U.S. and E.U. Approaches Lead to Fairness and Accountability?

AI@Northwestern and Northwestern University Law and Technology Initiative

Thursday, March 10, 2022

Recording

Resources from panelist Jeanna Matthews(Clarkson University/ACM/IEEE)

ACM ACM Technology Policy Council ACM US-Technology Policy Council	ACM US Public Policy Council (USACM) , Statement on Algorithmic Transparency and Accountability , January 12 2017. S. Garfinkel, J. Matthews, S. Shapiro, J. Smith Toward Algorithmic Transparency and Accountability , Communications of the ACM , Vol. 60, No. 9, Page 5, Sept. 2017, 10.1145/3125780. Panel session at National Press Club (9/14/2017), Summary . R. Baeza-Yates, T. Chen, N. Diakopoulos, A. Matthews, J. Matthews, E. Moss, A. Rosenthal , ACM US-TPC Comments on NIST Artificial Intelligence Risk Management Framework, January 27 2022. Comments relative to: NIST Concept Paper , Previous Comments, and Workshop
IEEE IEEE-USA AI Policy Commmitee	Privacy, Equity and Justice in Artificial Intelligence , Policy Statement, November 2021. Democratic Use of Artificial Intelligence , Policy Statement, November 2021. Artificial Intelligence: Job, Education, Workforce and Diversity , Policy Statement, November 2021. Ethically Aligned Design Effective Governance of Artificial Intelligence , Policy Statement, June 2021. IEEE 1012-2016 , IEEE Standard for System, Software, and Hardware Verification and Validation IEEE 829-2008, IEEE Standard for Software and System Test Documentation. Gabriela Bar, Gabriela Wiktorzak, Jeanna Matthews, Four Conditions for Building Trusted AI Systems: Effectiveness, Competence, Accountability, and Transparency IEEE Beyond Standards, July 13 2021.
Criminal Justice Software Accountability	Mats Heimdahl and Jeanna Matthews, Amici Curiae Brief in New Jersey v. Pickett , October 14 2020. Full Decision , Some key quotes Declaration in US v. Ellis, Heimdahl and Matthews, November 16 2020. Response, Nathan Adams and Jeanna Matthews, February 16 2021. Decision , Some key quotes S. Lacambra, J. Matthews and K. Walsh. Opening the Black Box: Defendants' Rights to Confront Forensic Software The Champion , May 2018. PDF J. Matthews, G. Northup, I. Grasso, S. Lorenz, M. Babaeianjelodar, H. Bashaw, S. Mondal, A. Matthews, M. Njie, J. Goldthwaite When Trusted Black Boxes Don't Agree: Incentivizing Iterative Improvement and Accountability in Critical Software Systems Proceedings of the 2020 AAAI/ACM Conference on Artificial Intelligence, Ethics and Society (AIES) , New York, New York, USA, February 7-8 2020. PDF (Paper), Slides J. Matthews, N. Adams, J. Goldthwaite Decoding Probabilistic Genotyping Software, Questioning Forensics 2020, 22 and You: Fighting for Privacy & Justice in an Age of Genetic Surveillance, Brooklyn Law School, Brooklyn, New York, USA, January 14-25 2020. N. Adams, S. Lorenz, M. Babaeianjelodar, J. Matthews, D. Krane Quantifying the impact of post-validation modifications to Forensic Statistical Tool Criminalistics Track, American Academy of Forensic Sciences (AAFS) 2019 Annual Scientific Meeting , February 18-23 2019. Abstract: PDF J. Matthews, S. Lorenz, M. Babaeianjelodar, A. Matthews, M. Njie, N. Adams, D. Krane, J. Goldthwaite, C. Hughes The Right To Confront Your Accusers: Opening the Black Box of Forensic DNA Software Proceedings of the 2019 AAAI/ACM Conference on Artificial Intelligence, Ethics and Society (AIES) , Honolulu, Hawaii, January 27-28 2019. PDF Brown Institute Magic Grant, Decoding Differences in DNA Forensic Software , 2018-2019. Magic Grant Profile Example of impact: STRmix ruled inadmissable in US v. Gissantaner, October 16 2019. ( PDF) J. Matthews, N. Adams, J. Greco, You're just complaining because you're guilty: A DEF CON Guide to Adversarial Testing of Software Used In the Criminal Justice System DEF CON 26 , Las Vegas, August 9-12 2018. Schedule , Slides , Video J. Matthews, N. Adams, J. Greco, You're just complaining because you're guilty: A Guide for Citizens and Hackers to Adversarial Testing of Software Used In the Criminal Justice System Bsides Las Vegas 2018, Las Vegas, August 7-8 2018. Video Michael Edge, Jeanna Matthews, Open practices in our science and our courtrooms , Open Science Framework, July 14 2021.
Some Other Resources	J. Matthews, Patterns and Anti-Patterns, Principles and Pitfalls: Accountability and Transparency in AI Association for the Advancement of Artificial Intelligence (AAAI) AI Magazine , April 2020. PDF (unformatted preprint, for beautiful PDF use link above) Catherine Muñoz, Jeanna Matthews, Jorge Pérez, Sistemas de toma de decisiones automatizadas: ¿De qué hablamos cuando hablamos de transparencia y del derecho a una explicación? Bits de Ciencia, Issue 21, pp. 27-36, 2021. Full Issue PDF Isabella Grasso, David Russell, Abigail Matthews, Jeanna Matthews, and Nicholas Record. Applying Algorithmic Accountability Frameworks with Domain-specific Codes of Ethics: A Case Study in Ecosystem Forecasting for Shellfish Toxicity in the Gulf of Maine Proceedings of the ACM-IMS Foundations of Data Science Conference(FODS-2020), October 18-20 2020. PDF Slides Christoph Ebell, Ricardo Baeza-Yates, Richard Benjamins, Hengjin Cai, Mark Coeckelbergh, Tania Duarte, Merve Hickok, Aurelie Jacquet, Angela Kim, Joris Krijger, John MacIntyre, Piyush Madhamshettiwar, Lauren Maffeo, Jeanna Matthews, Larry Medsker, Peter Smith, Savannah Thais Towards intellectual freedom in an AI Ethics Global Community AI and Ethics Journal , Springer, April 13 2021. R. Caplan, J. Donovan, L. Hanson, and J. Matthews Algorithmic Accountability: A Primer , Data and Society Report, April 18 2018. D. Roselli, J. Matthews, N. Talagala Managing Bias in AI Proceedings of the 1st Workshop on Fairness, Accountability, Transparency, Ethics, and Society on the Web , In Conjunction with the Web Conference 2019 San Francisco, CA, May 13-14 2019. PDF

ACM
ACM Technology Policy Council
ACM US-Technology Policy Council

ACM US Public Policy Council (USACM) ,
Statement on Algorithmic Transparency and Accountability ,
January 12 2017.

S. Garfinkel, J. Matthews, S. Shapiro, J. Smith
Toward Algorithmic Transparency and Accountability ,
Communications of the ACM , Vol. 60, No. 9, Page 5, Sept. 2017, 10.1145/3125780.
Panel session at National Press Club (9/14/2017), Summary .

R. Baeza-Yates, T. Chen, N. Diakopoulos, A. Matthews, J. Matthews, E. Moss, A. Rosenthal ,
ACM US-TPC Comments on NIST Artificial Intelligence Risk Management Framework, January 27 2022.
Comments relative to: NIST Concept Paper , Previous Comments, and Workshop

IEEE
IEEE-USA AI Policy Commmitee

Privacy, Equity and Justice in Artificial Intelligence , Policy Statement, November 2021.

Democratic Use of Artificial Intelligence , Policy Statement, November 2021.

Artificial Intelligence: Job, Education, Workforce and Diversity , Policy Statement, November 2021.

Ethically Aligned Design

Effective Governance of Artificial Intelligence , Policy Statement, June 2021.

IEEE 1012-2016 , IEEE Standard for System, Software, and Hardware Verification and Validation

IEEE 829-2008, IEEE Standard for Software and System Test Documentation.

Gabriela Bar, Gabriela Wiktorzak, Jeanna Matthews,
Four Conditions for Building Trusted AI Systems: Effectiveness, Competence, Accountability, and Transparency
IEEE Beyond Standards, July 13 2021.

Criminal Justice Software Accountability

Mats Heimdahl and Jeanna Matthews,
Amici Curiae Brief in New Jersey v. Pickett , October 14 2020. Full Decision , Some key quotes
Declaration in US v. Ellis, Heimdahl and Matthews, November 16 2020. Response, Nathan Adams and Jeanna Matthews, February 16 2021. Decision , Some key quotes

S. Lacambra, J. Matthews and K. Walsh.
Opening the Black Box: Defendants' Rights to Confront Forensic Software
The Champion , May 2018.
PDF

J. Matthews, G. Northup, I. Grasso, S. Lorenz, M. Babaeianjelodar, H. Bashaw, S. Mondal, A. Matthews, M. Njie, J. Goldthwaite
When Trusted Black Boxes Don't Agree: Incentivizing Iterative Improvement and Accountability in Critical Software Systems
Proceedings of the 2020 AAAI/ACM Conference on Artificial Intelligence, Ethics and Society (AIES) , New York, New York, USA, February 7-8 2020.
PDF (Paper), Slides

J. Matthews, N. Adams, J. Goldthwaite
Decoding Probabilistic Genotyping Software,
Questioning Forensics 2020, 22 and You: Fighting for Privacy & Justice in an Age of Genetic Surveillance, Brooklyn Law School, Brooklyn, New York, USA, January 14-25 2020.

N. Adams, S. Lorenz, M. Babaeianjelodar, J. Matthews, D. Krane
Quantifying the impact of post-validation modifications to Forensic Statistical Tool Criminalistics Track, American Academy of Forensic Sciences (AAFS) 2019 Annual Scientific Meeting , February 18-23 2019.
Abstract: PDF

J. Matthews, S. Lorenz, M. Babaeianjelodar, A. Matthews, M. Njie, N. Adams, D. Krane, J. Goldthwaite, C. Hughes
The Right To Confront Your Accusers: Opening the Black Box of Forensic DNA Software
Proceedings of the 2019 AAAI/ACM Conference on Artificial Intelligence, Ethics and Society (AIES) , Honolulu, Hawaii, January 27-28 2019.
PDF

Brown Institute Magic Grant,
Decoding Differences in DNA Forensic Software , 2018-2019.
Magic Grant Profile
Example of impact: STRmix ruled inadmissable in US v. Gissantaner, October 16 2019. ( PDF)

J. Matthews, N. Adams, J. Greco,
You're just complaining because you're guilty: A DEF CON Guide to Adversarial Testing of Software Used In the Criminal Justice System
DEF CON 26 , Las Vegas, August 9-12 2018.
Schedule , Slides , Video

J. Matthews, N. Adams, J. Greco,
You're just complaining because you're guilty: A Guide for Citizens and Hackers to Adversarial Testing of Software Used In the Criminal Justice System
Bsides Las Vegas 2018, Las Vegas, August 7-8 2018.
Video

Michael Edge, Jeanna Matthews,
Open practices in our science and our courtrooms ,
Open Science Framework, July 14 2021.

Some Other Resources

J. Matthews,
Patterns and Anti-Patterns, Principles and Pitfalls: Accountability and Transparency in AI
Association for the Advancement of Artificial Intelligence (AAAI) AI Magazine , April 2020.
PDF (unformatted preprint, for beautiful PDF use link above)

Catherine Muñoz, Jeanna Matthews, Jorge Pérez,
Sistemas de toma de decisiones automatizadas: ¿De qué hablamos cuando hablamos de transparencia y del derecho a una explicación?
Bits de Ciencia, Issue 21, pp. 27-36, 2021.
Full Issue PDF

Isabella Grasso, David Russell, Abigail Matthews, Jeanna Matthews, and Nicholas Record.
Applying Algorithmic Accountability Frameworks with Domain-specific Codes of Ethics: A Case Study in Ecosystem Forecasting for Shellfish Toxicity in the Gulf of Maine
Proceedings of the ACM-IMS Foundations of Data Science Conference(FODS-2020), October 18-20 2020.
PDF
Slides

Christoph Ebell, Ricardo Baeza-Yates, Richard Benjamins, Hengjin Cai, Mark Coeckelbergh, Tania Duarte, Merve Hickok, Aurelie Jacquet, Angela Kim, Joris Krijger, John MacIntyre, Piyush Madhamshettiwar, Lauren Maffeo, Jeanna Matthews, Larry Medsker, Peter Smith, Savannah Thais
Towards intellectual freedom in an AI Ethics Global Community
AI and Ethics Journal , Springer, April 13 2021.

R. Caplan, J. Donovan, L. Hanson, and J. Matthews
Algorithmic Accountability: A Primer ,
Data and Society Report, April 18 2018.

D. Roselli, J. Matthews, N. Talagala
Managing Bias in AI
Proceedings of the 1st Workshop on Fairness, Accountability, Transparency, Ethics, and Society on the Web , In Conjunction with the Web Conference 2019 San Francisco, CA, May 13-14 2019.
PDF