I also designed an A|B test to find these issues in the remaining puzzles. There are six  puzzles in the game, each with one solution. I wanted to measure how long it took players to solve each puzzle (how long it took them to find and combine the correct items, not including the time it took them to deliver it to the requestor) and how many incorrect combination attempts they made per puzzle. I specifically wanted to test how the placement of objects in the level affected playtime. I thought this could correspond to the difficulty of finding hidden objects in the space. Measuring the incorrect combination attempts was an attempt to understand strategy and player interpretation. Since the ‘correct’ adjective of an object in the game was revealed to the player if they combined the object incorrectly with a non-requested quality, I wanted to see if in general players used this to learn about the world quickly (and if so at what point in the game.) I also thought this might correspond to players not understanding the quality of an object they were supposed to find. For example, do they interpret a broom as “clean” or “dirty”?

​

Build A was our ‘control’ using the item placements the team had designed as part of our gatekeeping strategy to get the player to explore the entire apartment complex. Build B had more carefully considered item placement in which sightlines, color, framing, and the composition of other objects in the room drew players toward items.