UCL School of Management is delighted to welcome Georg Rilinger, MIT, to host a research seminar discussing; How The Experimental Hand Errs. 

Abstract:

Experimentation is often regarded as a strategic tool for organizational decision-making, yet platform companies face unique challenges in leveraging experiments effectively. Despite unparalleled capacities for experimentation to inform design decisions, platform organizations frequently suffer from platform degeneration and incrementalism in design changes. This study investigates the understudied organizational dynamics behind experimental decision-making, focusing on iLabour, a digital labor platform. Based on 12 months of ethnographic fieldwork and archival research, we analyze the implementation of project MATCH, an initiative to redesign algorithmic matching between customers and contractors. The findings reveal that designers working on different aspects of the platform had conflicting priorities, with initial support for systemic change giving way to fragmentation. Early experiments created ‘data politics,’ where groups with straightforward evidence of user preferences for the status quo prevailed over those presenting ambiguous experimental findings. As a result, product managers were unable to retract or iterate prototypes as envisioned, leading to the entrenchment of incomplete designs within the platform’s architecture. This study shows how competing interests and fragmented data-driven processes undermine experimental ideals, resulting in inefficient and incremental platform changes. By highlighting the role of occupational dynamics and data politics, it contributes to understanding technology, expertise, and organizational design in platform marketplaces.