The conversational interface has been commercialized in various forms, from assistants built into personal devices to agents in the form of artificial intelligence (AI) speakers. An agent in the form of an AI speaker is often used by multiple users in a physically shared space. Because of changes in using environments, research on conversational agents is mostly conducted on single-user personal assistants. In this project, we investigate the user experience on an AI speaker that embodies an AI assistant in a multi-user home environment. We aim to report findings of interesting experiential patterns of with-use situations and user perceptions on the AI assistant in a multi-user home environment.