Automating tasks can make a smartphone easier to use and even more battery efficient. However, little work has been done on empowering end users to create such automations. In this research we propose an approach for automating smartphone tasks by retrospective demonstration. Succinctly, we consider the logic behind the approach as “keep doing what I just did”: the automation app continuously records the users’ interactions with their phones, and after users perform a task that they would like to automate, they can ask the app to create an automation rule based on their latest recorded actions. Since users only have to use their smartphones, as they would naturally do, to demonstrate the actions, we believe that this approach can lower the barrier for creating smartphone automations.

To evaluate our approach, we developed prototypes of an app called Keep Doing It, which supports automating tasks by demonstration. We conducted a lab user study with the first prototype to gather participants’ first impressions. The participants created automation rules using our application based on given scenarios. Based on their feedback and on our observations, we refined the prototype and conducted a five-day remote user study with new participants, who could then create which and how many rules they wanted. Overall, the findings of both studies suggest that, although there were some occasional inaccuracies, participants would be willing to automate smartphone tasks by demonstration due to its ease of use. We concluded that this approach has much potential to aid end users to automate their smartphones, but there are still issues that need to be addressed by further research.

Some of the topics researched were context-aware systems, end-user development, programming by demonstration and smartphone automation.