Control Android without ADB: MCP, Tailscale and local AI

You can control Android without ADB by exposing the actions you need from the phone itself. I built Mi Android MCP, a native Java Android app that reads screen state, takes screenshots and interacts with other apps through Android accessibility. Control requests travel over a private Tailscale network, with the option to connect a local Ollama model.

I tested it on a real Moto g04 running Android 14, controlling Reddit and other apps through Tailscale without using ADB. This article explains the project's architecture and setup: Android automation without USB, wireless debugging or a paid AI provider.

What is MCP, and how does it help with Android automation?

MCP stands for Model Context Protocol. It lets a client application discover and use tools exposed by a server. A model proposes an action; the software connecting it to the server requests execution. The MCP architecture documentation explains this separation between clients, servers and tools.

In my project, the server runs inside the phone. MCP exposes nine specific tools, such as reading screen state or pressing Back. It is neither the AI model nor a replacement for Android permissions: the ability to act comes from the accessibility service enabled by the user.

Why control Android without ADB?

ADB is a useful development tool. My reason for leaving it out here is operational: I want to control a phone from a PC over a private network without maintaining a debugging connection.

Common workflows involving ADB, scrcpy or Appium have different purposes and requirements. For context, the ADB documentation covers USB and wireless debugging; scrcpy provides screen mirroring and control; and Appium's UiAutomator2 driver supports automation and testing.

There is no need to dismiss ADB to choose a different architecture. I removed that dependency because the operations I need fit within the accessibility service. For a testing-oriented approach, see my practical guide to Appium for Android.

Architecture: from the phone to the local model

The solution has five parts with distinct responsibilities:

1. Android phone: runs the apps I want to control.
2. Android accessibility: provides interface inspection and the available actions.
3. Mi Android MCP: native app with package es.david.androidmcp, hosting an HTTP MCP server on port 8765.
4. Private Tailscale network: connects the computer to the phone's private IP.
5. Android local: Python launcher on the computer that connects Ollama to the phone's MCP server.

For screen reading, the chain is phone → accessibility → MCP server → Tailscale → Python launcher → local model. An action travels in the opposite direction and is executed by the accessibility service.

Accessibility is the only control mechanism

The app uses the accessibility service to read the screen, take screenshots, tap, swipe, enter text, press Back or Home, and open apps. Android documents these capabilities and their conditions in the AccessibilityService reference.

This does not grant unrestricted access to the phone. Available interface information depends on what each app exposes to accessibility, and Android can restrict screenshots of protected content. A screen that is difficult to interpret remains a limitation the automation must handle.

MCP and Ollama are separate components

The Android server provides tools; the model selects an action based on the state it receives. I can test the tools through an MCP client before adding AI to the workflow. This helps distinguish a connection problem from an incorrect model decision.

The nine available MCP tools

This table describes capabilities, not argument contracts. To construct calls, use the schema exposed by each tool in the installed version.

This MCP server does not include Appium, XPath, ADB or deep links. It does not transfer files or generate 2FA codes. Opening an app does not mean it can jump directly to any internal screen.

Security: private network, Bearer token and origin validation

The server binds only to the private Tailscale IP. It does not listen on every phone interface or require a public router port. Tailscale provides private connectivity and encrypts traffic between devices, as described in its documentation.

The server also requires a Bearer access token and applies origin validation. These controls work together: the network restricts where connections can come from, while the token authenticates requests. Origin validation adds a check; it does not replace the token or network restrictions.

HTTP is the server transport; in this design, traffic between the computer and phone travels inside Tailscale. Remote control does not require turning that private address into a public endpoint.

Accessibility permission allows interaction with the interface. Grant it only to the app you intend to install, and keep the token out of screenshots, repositories and shared logs.

Step-by-step setup

This guide assumes you have the project's app and launcher. It describes the actual configuration without assuming a public download or an installer that is not documented here.

1. Install Mi Android MCP on the phone

Install package es.david.androidmcp using Android's package installer. Neither installation nor subsequent control requires ADB. The test described here used a Moto g04 with Android 14; it is not a compatibility certification for every manufacturer and version.

2. Enable the accessibility service

In the phone's accessibility settings, enable the Mi Android MCP service. Its location and any prompts can vary by device and installation method. Check that Android keeps the service enabled before trying to execute actions.

3. Connect the phone and PC to Tailscale

Connect both devices to your private network and check that its access policy allows the computer to reach the phone. The server must use the phone's Tailscale IP and port 8765, rather than its Wi-Fi IP or a public address.

4. Connect the MCP client and check the tools

With the server running, configure the client with the project's MCP address, Bearer token and an origin accepted by the server. Use the HTTP path and configuration from the installed version: the port alone does not specify the endpoint path.

First, check that all nine tools appear. Then request android_get_screen_state and try a simple action such as android_home. Read the state again to confirm the result before automating a complete workflow.

5. Prepare Ollama and Android local on the computer

With Ollama installed, download the example model:

ollama pull qwen3:4b

Configure the Android local Python launcher to use that model and the phone's MCP connection. The model runs on the computer; the control server runs on the phone. The initial download needs connectivity, and response speed depends on hardware and context.

This workflow does not require a paid AI API. It still consumes computing resources, and it does not make Reddit or other internet-dependent apps work offline.

6. Start with a small, verifiable task

Begin by opening an app, reading the screen and returning Home. Then add a swipe or text entry. Inspect the state after every action: a completed call does not by itself prove that the interface has reached the intended result.

How to automate a phone with local AI

Android local follows a cycle: read → decide → execute → verify. It reads state, asks the model for a structured JSON decision, executes the action through MCP and checks the screen again. It includes recovery when progress stalls and protection against repeated actions.

Ollama can constrain response format with a JSON schema, as explained in its structured outputs guide. A correctly formatted response can still choose the wrong action, so verification matters as much as format.

Using qwen3:4b here does not imply that it visually interprets screenshots. Reading accessibility state and capturing a screenshot are separate capabilities. I also avoid inventing a callable JSON example: its fields must match what the launcher actually accepts.

Repetition protection matters when the screen does not change. If the model keeps proposing the same action, the system needs to handle the stalled state instead of accumulating taps. Recovery does not guarantee that every interface or interruption can be resolved.

Practical example: navigating Reddit without ADB

I tested Reddit control on the Moto g04 running Android 14 over Tailscale. The following simple sequence illustrates how the tools fit together; it is not a verbatim execution log:

1. Open Reddit with android_open_app.
2. Call android_get_screen_state to identify the visible screen.
3. Identify an available navigation control and select it with android_click_element.
4. If the workflow includes a search field, focus it and use android_type_text.
5. Swipe with android_swipe when needed, then read the screen again.
6. Use android_screenshot if a screenshot is needed to review the state.
7. Return with android_back or finish at the home screen with android_home.

The specific controls depend on the Reddit version and current screen. Automation needs to observe that state at each step. The test demonstrates control of a real device in that configuration; it provides neither a measured success rate nor a guarantee that every workflow can run unattended.

What to check before commissioning an Android pilot

I would first check whether the app offers an API that covers the process. If it does, I would compare that integration with maintaining a workflow that depends on screens. This approach is worth testing when the work needs to happen in the app and accessibility exposes the required actions.

MCP does not require a model to choose every step. For a stable journey, I would start with a controlled sequence and add AI only where it contributes a useful decision. My guide to agents versus workflows examines that choice.

Before adding devices, I would propose these checks for the specific combination of phone, Android version, app and MCP client. They are acceptance criteria for a future engagement, not measured results from this project:

For a hypothetical order-status lookup, the goal would be to retrieve the correct status and retain evidence. Opening the app would not be enough. If the order disappears or the session expires, the outcome should await review rather than be marked as resolved.

I would separate tasks completed without help, tasks recovered by a person and failed tasks. I would also agree who maintains the workflow when the app changes. The SwimUpvote case documents queues and traceability in another platform; it does not establish this MCP server's capacity or compatibility with a device fleet.

Questions to decide whether it fits

Does MCP replace Appium?

They serve different roles: MCP exposes tools, while Appium automates applications. The Appium project even provides an MCP server. My implementation uses accessibility on the phone and does not include Appium. The choice depends on the workflow and the tests you need.

Does it work on every Android phone?

The documented test used a Moto g04 running Android 14. I would validate each device and app combination before including it in scope. An accessible interface in one version can change in the next.

Do I need a paid AI service or a public download?

The workflow described here runs Ollama on the computer and needs no paid AI API. It still requires the hardware, the project's app and its launcher. This page does not offer a public download; access to those components needs to be clarified before planning an implementation.

From this implementation to your workflow

To scope a pilot, I would need the app, phone models, manual journey, allowed actions and the result you would accept. A proposal should separate initial validation, implementation and maintenance; use my software quote comparison guide to review those items.

To apply this approach to a business process, explore my automation and applied AI services or tell me about the workflow you need to automate. I can review whether an integration, a screen-based workflow or a limited trial makes sense before expanding the scope.