In the First Part of This Series I wrote about an incredible feature called Computer Use that Anthropic released a few weeks ago. This could be a bombshell for the IT industry and many other sectors (I’m afraid the resonance is yet to come).

Overview

Part I: The announcement and the early implications on the economy
Part II: A real-life example — Letting Claude accomplish a daunting task all by himself
Part III: Do it yourself — Quickstart guide for non-techies, tips and tricks

In the second part of the blog post, I will take you through a real-world scenario that demonstrates how Computer Use can automate the process of booking a doctor’s appointment. We’ll cover understanding the task, introducing Computer Use, setting up the application, executing the task, and summarizing the experience along with future implications.

Although we are still in the early days, this technology is, in my humble opinion, the first truly capable tool to perform a task from A to Z with very little human intervention. The autonomy and ease of use are amazing—something I’ve never seen before. And believe me, I’ve tried a lot! 😄

To go beyond the theory and the news, I decided to share with you a real-life scenario—finding and booking a doctor’s appointment using Claude Computer Use.

If you don’t already know how it works, I suggest you read my previous articles on the subject. But here again, in a highly compressed form:

Computer Use is a new feature of the Claude model (like ChatGPT) that can understand user interfaces on your screen and issue commands to control your screen like any human would.

And Here Is Our Goal

"Search and book a doctor appointment on a website on Doctor.co.uk"

I use the appliance provided by Claude in a Docker container and then proceed as follows:

What I Did

1. Start the Appliance Provided by Anthropic

First, I started the appliance.

2. Configure the Appliance

Then I configured it to use Claude from AWS Bedrock because of token limitation when I use Anthropic Claude (more on this in the next part of this blog series).

3. Let the Magic Begin

Having done that, now the magic can begin by simply prompting the model as follows. This was also the only instruction I had to give Claude to accomplish this final goal.

Book a general practice appointment any time this month after 4pm at doctap.co.uk. Jane Smith
Date of Birth: 1990-04-25
Gender: Female
Address Line 1: 456 Elm Street
Address Line 2: Apt 12B
City: Manchester
Postcode: M1 2AB
Contact Number: +44 7911 123456
Email Address: jane.smith@example.com

And here is what happened with zero intervention from my side. Note that all the messages below are mostly information until the point it asked for my confirmation to book a suitable appointment it found.

And a few minutes later, coming back from a coffee break… Task almost finished!

In Summary Claude:

• Filtered by location
• Filtered by doctors (general practitioner)
• Navigated through the results
• Inspected the viable time slots
• Confirmed a matching slot
• Started the booking process
• Only in the last step, however, it encountered a safety restriction. If you’re already registered and logged in as mentioned earlier, this shouldn’t be a problem anymore.

Remember that Claude had no prior knowledge about the whole environment — neither the OS, the browser, nor the website. It figured out all by itself just like a human would do by analyzing images and having a goal in mind.

Just amazing.

Limitations

As with any new technology, there are a few limitations we have to take into consideration (though some of them will be obsolete soon with the current pace).

• Anthropic has hard token limit sizes for minute and daily limits. This could be cumbersome after some time, especially if you transfer all 10 screenshots with each iteration. Bedrock, on the other hand, does not have such rigid limitations, which is why I always use it for Computer Use (though higher cost because of lack of prompt caching).
• The virtualization solution provided in this appliance could take a significant amount of memory. Pay attention that you don’t hit a memory limit on your device in the middle of a session, with possible loss of the whole conversation.
• Anthropic Claude has support for prompt caching, which can significantly reduce the input token usage cost (at the cost of daily limits). Bedrock does not have prompt caching yet, which leads to more cost over time. It’s up to you to decide what is more important for you (either try again when you hit the limit and pay less or just let it do its thing and pay more).
• Security restrictions may occur during steps like registration for obvious reasons, as noted earlier. Without restrictions, people could create numerous fake accounts, which is harmful to everyone.
• If the conversation gets too long, you might reach a context limit, as you may know from Claude Web. You can extend this limit by reducing the screenshots during sessions and by asking Claude to save lengthy results to files instead of displaying them in the chat.

Use Cases

As I mentioned in the first part, full autonomous computer automation has and will remain the holy grail in AI, if not the whole economy, because of the enormous potential for cost and time-saving across all industries.

Though there are many use cases where computer use, even in its best efficiency, won’t be an option (like in high-frequency machine-to-machine interaction via API), if you understand how Computer Use works and consider its limitations, the possibilities are endless.

Here are just a few ideas from Claude/GPT that are worth trying.

You can also try to get some ideas for your occupation here: https://udify.app/chat/y0kbR2jfjvw7Y7ah (no warranties, of course)

• Automating E-commerce Shopping: Automatically find the best deals for a product on multiple websites. Compare prices and available discounts, then add the item to the cart and checkout.
• Scheduling Appointments: Automate the process of scheduling appointments for haircuts, fitness classes, or other personal services. Search for available times and book the slot that fits your schedule.
• Email Management: Automate inbox cleanup by deleting unwanted emails, archiving old threads, or marking specific emails as important. Set up filters to automatically categorize incoming messages.
• Social Media Content Scheduling: Use the technology to schedule posts on platforms like Instagram, Twitter, or Facebook. Automate responses to messages or comments based on specific triggers.
• Form Filling: Automatically fill in online forms (job applications, surveys, or registration forms) using pre-filled data or templates.
• Hotel or Flight Booking: Automate booking processes for travel (e.g., flights, hotels) based on a set of preferences (location, price range, time).
• Data Extraction from Websites: Extract data from various websites (such as news sites, real estate listings, or research papers) and compile it into a structured format for analysis.
• Online Research Tasks: Automate the process of gathering information about a specific topic (e.g., by scraping multiple resources) and summarizing it in a document.

Wrap Up

This piece of technology is just the beginning, but it is already so powerful that it as it advances, it could literally save us an incredible amount of time on boring tasks like booking a doctor’s appointment, finding a flight and a whole lot more.

If you want to see Computer Use in action with your use case, I highly recommend you give it a try yourself.

Has Anthropic Claude just wiped out an entire industry? (Part 3/3)

Discovering the Latest in AI: Trends, Tools, and Tutorials

AI Rabbit News & TutorialsAiRabbit