How to integrate Google Gemini with any Apple device

Apple, while known for its innovation, has been somewhat slow in adopting new technologies such as foldable displays and touch screens on Macs. One area where this lag is particularly noticeable is in the integration of generative AI (GenAI), which has been widely available for a few years now. This tutorial aims to empower you to break free from these constraints and build your own AI solution on your Mac. This solution will be accessible from anywhere and adaptable to any model, prompt, or use case.

For this tutorial, we'll be integrating Gemini. While it might not be the best for coding or reasoning, its writing capabilities are invaluable for everyday tasks like:

• Summarizing
• Rephrasing
• Translation

Want to See It in Action?
If you're short on time or prefer a quick solution, you can simply download the shortcut and have it running in just 2 minutes. By using the shortcut, you'll be directly supporting our efforts to bring these ideas to life.

How it works

The concept is simple (though a bit tricky to implement):

1. Read Clipboard Content: The workflow will read the content from your clipboard.
2. Call Gemini API: It will then call the Gemini API with a system prompt that you can customize at any time. This flexibility is the beauty of open solutions.
3. Display Results: The workflow will display the results to you, allowing you to confirm if they meet your expectations.
4. Copy to Clipboard: The results will be copied back to your clipboard.
5. Paste Anywhere: You can then paste the clipboard content wherever you need it.

To make this solution accessible from anywhere on your Mac and any Apple device, we'll add it to the services menu with a simple checkbox.

Let's Get Started!

Step 1: Create a New Shortcut Workflow

Start by creating a new Shortcut workflow.

Step 2: Name Your Workflow "Ask Gemini"

Give your new workflow a descriptive name, "Ask Gemini."

Step 3: Configure the Service Menu

In the details tab, select the "Service" menu option. This will allow you to access your workflow from anywhere on your Mac. You can also assign a keyboard shortcut later for even quicker access.

Make sure to let the workflow read from the Clipboard if it is not started with selected text.

One note on that: you can trigger the workflow in two ways:By simply copying a piece of rich text or plain text to the clipboard and then starting the shortcut. The result is copied back to the clipboard, so you simply have to paste the result after a quick verification. This is the preferred approach almost everywhere, except in Apple Notes. Here the second, even more convenient way comes into play.In Apple Notes, you can select any text, right-click, and choose Services -> Ask Gemini. This will pass the selected text automatically to the workflow, and the result will replace the selected text automatically. This, however, works only in a few Apple apps like Apple Notes on Mac (on iPhone, it does not work).

Step 4: Convert Markdown to Plain Text

Since you'll likely be copying and pasting markdown text, it's essential to convert it to plain text for Gemini to understand. Add a "Convert Markdown to Plain Text" action to your workflow.

Step 5: Add a Text Box for System Prompts

Include a "Text" action to create a text box for your system prompts. This allows you to customize your requests to Gemini.

Assign a variable to the system prompt as depicted below.

Let's say you want to summarize the text, you could add something like this. Remember that the main content will be passed as an input parameter to the shortcut and will be referred to as "given text."

summarize the given text

Or translate to English:

translate the given text to English.

Or think of any prompt you like.

Step 6: Prepare the API Endpoint for Gemini

First, you need to get an API key. You can create one for free here.

Now get back to the shortcut and create another text node and place your API key here.

https://generativelanguage.googleapis.com/v1beta/models/gemini-1.5-flash:generateContent?key=<YOUR API KEY HERE>

Now we can make the actual call to Gemini by making an HTTP request.

Create a new action like this:

And make sure the structure of the request is 1:1 like below, otherwise it will not work. This one is a bit tedious, I know.

Step 7: Extract the Response Text

After the request has been made, we need to get only the response text from the response. For this, we need an action that extracts the text result from the response.

Enter the value exactly as depicted below.

Step 8: Convert Back to Rich Text

Now we have the result. And remember, if this is in markdown, we need to convert it back to rich text so you can use it where you like (including images!).

For this, we add another action for the reverse conversion.

Step 9: Output the Result

Finally, output the result using a stop and output action. As mentioned above: in case it is in one of the Apple-supported text editors like Apple Notes, it will simply replace the selected text. In all other cases, it will copy the result to the clipboard. Here is how Apple explains this action:

Knowing that, let's add the final step to stop and output or copy to the clipboard as depicted below.

Now we can actually test it.

Open any Apple Note with any text and select the text. You should now be able to see the new "Ask Gemini" service.

In case you have other services, it will be added to other services under the service menu.

In case you want to empower text in, say, the browser, you simply copy the text to edit, hit that same service, and copy-paste back the results. That's it.

If you are interested in workflows for other models like OpenAI or Groq with multimodal capabilities, drop me an email and I will send you the link for download.

With this shortcut, you have the full flexibility to embed all the power of Gemini anywhere you have text on your Mac and Apple devices.

In the next part, we will continue with multimedia applications.

Stay tuned.

Feel free to reach out with any questions or feedback!