Data Leverage References

Tan, Joshua, Vincent, Nicholas, Elkins, Katherine, Sahlgren, Magnus

Arslan Akram

Choe, Sang Keun and Ahn, Hwijeen and Bae, Juhan and Zhao, Kewen and Kang, Minsoo and Chung, Youngseog and Pratapa, Adithya and Neiswanger, Willie and Strubell, Emma and Mitamura, Teruko and Schneider, Jeff and Hovy, Eduard and Grosse, Roger and Xing, Eric

{NISO}

Johnson, Isaac, Kaffee, Lucie-Aim{\'e}e, Redi, Miriam

Liu, Jason

Explore how data flywheels leverage user feedback to enhance product development and achieve business success with AI.

Anton Lozhkov, Raymond Li, Loubna Ben Allal, Federico Cassano, Joel Lamy-Poirier, Nouamane Tazi, Ao Tang, Dmytro Pykhtar, Jiawei Liu, Yuxiang Wei, Tianyang Liu, Max Tian, Denis Kocetkov, Arthur Zucker, Younes Belkada, Zijian Wang, Qian Liu, Dmitry Abulkhanov, Indraneil Paul, Zhuang Li, Wen-Ding Li, Megan Risdal, Jia Li, Jian Zhu, Terry Yue Zhuo, Evgenii Zheltonozhskii, Nii Osae Osae Dade, Wenhao Yu, Lucas Krauß, Naman Jain, Yixuan Su, Xuanli He, Manan Dey, Edoardo Abati, Yekun Chai, Niklas Muennighoff, Xiangru Tang, Muhtasham Oblokulov, Christopher Akiki, Marc Marone, Chenghao Mou, Mayank Mishra, Alex Gu, Binyuan Hui, Tri Dao, Armel Zebaze, Olivier Dehaene, Nicolas Patry, Canwen Xu, Julian McAuley, Han Hu, Torsten Scholak, Sebastien Paquet, Jennifer Robinson, Carolyn Jane Anderson, Nicolas Chapados, Mostofa Patwary, Nima Tajbakhsh, Yacine Jernite, Carlos Muñoz Ferrandis, Lingming Zhang, Sean Hughes, Thomas Wolf, Arjun Guha, Leandro von Werra, Harm de Vries

Longpre, Shayne and Mahari, Robert and Lee, Ariel and Lund, Campbell and Oderinwale, Hamidah and Brannon, William and Saxena, Nayan and Obeng-Marnu, Naana and Sud, Tobin and Gupta, Sameer and Muennighoff, Niklas and others

Roche, Adam, Sassoon, Yali

Shen, Judy Hanwen, Raji, Inioluwa Deborah, Chen, Irene Y

Luca Soldaini, Rodney Kinney, Akshita Bhagia, Dustin Schwenk, David Atkinson, Russell Authur, Ben Bogin, Khyathi Chandu, Jennifer Dumas, Yanai Elazar, Valentin Hofmann, Ananya Harsh Jha, Sachin Kumar, Li Lucy, Xinxi Lyu, Nathan Lambert, Ian Magnusson, Jacob Morrison, Niklas Muennighoff, Aakanksha Naik, Crystal Nam, Matthew E. Peters, Abhilasha Ravichander, Kyle Richardson, Zejiang Shen, Emma Strubell, Nishant Subramani, Oyvind Tafjord, Pete Walsh, Luke Zettlemoyer, Noah A. Smith, Hannaneh Hajishirzi, Iz Beltagy, Dirk Groeneveld, Jesse Dodge, Kyle Lo

Raul Castro Fernandez

Moritz Hardt, Eric Mazumdar, Celestine Mendler-Dünner, Tijana Zrnic

Provides theoretical framework for algorithmic collective action, showing that small collectives can exert significant control over platform learning algorithms through coordinated data strategies.

Hanlin Li, Nicholas Vincent, Stevie Chancellor, Brent Hecht

Sung Min Park, Kristian Georgiev, Andrew Ilyas, Guillaume Leclerc, Aleksander Madry

Introduces TRAK (Tracing with the Randomly-projected After Kernel), a data attribution method that is both effective and computationally tractable for large-scale models by leveraging random projections.

{Common Crawl}

Andrew Ilyas, Sung Min Park, Logan Engstrom, Guillaume Leclerc, Aleksander Madry

Proposes datamodels that predict model outputs as a function of training data subsets, providing a framework for understanding data attribution through retraining experiments.

Yongchan Kwon, James Zou

Generalizes Data Shapley using Beta weighting functions, providing noise-reduced data valuation that better handles outliers and mislabeled data detection.

{LAION}

{LAION}

Schuhmann, Christoph, Beaumont, Romain, Vencu, Richard, Gordon, Cade, Wightman, Ross, Cherti, Mehdi, Coombes, Theo, Katta, Aarush, Mullis, Clayton, Wortsman, Mitchell, Schramowski, Patrick, Kundurthy, Srivatsa, Crowson, Katherine, Schmidt, Ludwig, Kaczmarczyk, Robert, Jitsev, Jenia

{BigCode Project}

Kwon, Yongchan and Zou, James

Alexandra Sasha Luccioni, Joseph D. Viviano

Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, Shawn Presser, Connor Leahy

Carlos Toxtli, Siddharth Suri, Saiph Savage

Nicholas Vincent, Brent Hecht

Vincent, Nicholas and Li, Hanlin and Tilly, Nicole and Chancellor, Stevie and Hecht, Brent

Jo, Emily, Gebru, Timnit

Sarah Meldrum, Sherlock A. Licorish, Bastin Tony Roy Savarimuthu

Garima Pruthi, Frederick Liu, Mukund Sundararajan, Satyen Kale

Introduces TracIn, which computes influence of training examples by tracing how test loss changes during training. Uses first-order gradient approximation and saved checkpoints for scalability.

Baumgartner, Jason, Zannettou, Savvas, Keegan, Brian, Squire, Megan, Blackburn, Jeremy

Crawford, Kate, Paglen, Trevor

Amirata Ghorbani, James Zou

Proposes data Shapley as a metric to quantify the value of each training datum to predictor performance, satisfying equitable data valuation properties from cooperative game theory.

Antoine Miech, Dimitri Zhukov, Jean{-}Baptiste Alayrac, Makarand Tapaswi, Ivan Laptev, Josef Sivic

Ruoxi Jia, Dah-Yuan Dao, Boxin Wang, Frances Ann Hubis, Nick Hynes, Neslihan M. Gurel, Carl J. Spanos

Pang Wei Koh, Kai-Siang Ang, Hubert H. K. Teo, Percy Liang

Nicholas Vincent, Yichun Li, Renee Zha, Brent Hecht

Nicholas Vincent, Brent Hecht, Shilad Sen

Simulates data strikes against recommender systems, showing that collective withholding of training data can create leverage for users against technology platforms.

Imanol Arrieta-Ibarra, Leonard Goff, Diego Jimenez-Hernandez, Jaron Lanier, E. Glen Weyl

Bender, Emily M., Friedman, Batya

Gebru, Timnit, Morgenstern, Jamie, Vecchione, Briana, Vaughan, Jennifer Wortman, Wallach, Hanna, Daumé III, Hal, Crawford, Kate

{International Internet Preservation Consortium}

Pang Wei Koh, Percy Liang

Uses influence functions from robust statistics to trace model predictions back to training data, identifying training points most responsible for a given prediction.

Aniket Kittur, Jeffrey V. Nickerson, Michael Bernstein, Elizabeth Gerber, Aaron Shaw, John Zimmerman, Matt Lease, John Horton

Kollock, Peter

The study of social dilemmas is the study of the tension between individual and collective rationality. In a social dilemma, individually reasonable behavior leads to a situation in which everyone is worse off. The first part of this review is a discussion of categories of social dilemmas and how they are modeled. The key two-person social dilemmas (Prisoner’s Dilemma, Assurance, Chicken) and multiple-person social dilemmas (public goods dilemmas and commons dilemmas) are examined. The second part is an extended treatment of possible solutions for social dilemmas. These solutions are organized into three broad categories based on whether the solutions assume egoistic actors and whether the structure of the situation can be changed: Motivational solutions assume actors are not completely egoistic and so give some weight to the outcomes of their partners. Strategic solutions assume egoistic actors, and neither of these categories of solutions involve changing the fundamental structure of the situation. Solutions that do involve changing the rules of the game are considered in the section on structural solutions. I conclude the review with a discussion of current research and directions for future work.

Marwell, Gerald, Oliver, Pamela

{arXiv.org}

{arXiv.org}

{arXiv.org}

{TensorFlow Datasets}

{Common Crawl}

{Common Crawl}

{OpenAI}

{École Normale Supérieure}

{NLM}

{jsonlines.org}

{Library of Congress}

Hendrycks, Dan

{ndjson}

{OpenAssistant}

{Apache Software Foundation}

{OpenAI}

{Project Gutenberg}

{Project Gutenberg}

{Pushshift}

{Reddit}

{Reddit Help}

{Stack Exchange}

{Meta Stack Exchange}

{BigCode Project}

{BigCode Project}

{BigCode Project}

{TensorFlow Datasets}

{TensorFlow}