The AI War Has Begun! Every Google I/O AI Announcement

Good morning

every

one, welcome to Google I O as an AI-first company. We are at an interesting turning point. Let me start with some examples of how generative AI is helping to evolve our products. Let's say you received this email stating that your flight was canceled and the airline sent it. a coupon, but what you really want is a full refund, you can reply and use, help me write, just type the message of what you want, an email to request a full refund, press create and a complete draft will appear, as you can see it conveniently extracted. in the flight details from the email above and it looks pretty close to what you want to send, you may want to refine it further.

In this case, a more elaborate email could increase your chances of getting the refund and that's it, I think you're ready to send. help me write, we will start implementing it as part of our workspace updates. The next example is Maps. Imagine if you could see your entire trip in advance with an immersive view of the routes. Now you can do it, whether you're walking, biking or driving, let me show you. What I mean is that I'm in New York City and I want to go for a bike ride. Maps has given me a couple of options close to where I am.

I like the one on the boardwalk, so let's go with the one that looks scenic. I want to Feel it first, click on the immersive route view and it's a whole new way to see my trip. I can zoom in to get an amazing panoramic view of the course and as we turn around we come to a great bike path. and if I want to check the traffic and weather and see how they might change in the next few hours I can do that, it looks like it's going to pour rain later so I might want to start now.

The immersive view of the routes will begin to roll out in the summer and launch in 15 cities before the end of the year another product improved thanks to AI is Google Photos

every

month. 1.7 billion images are edited in Google Photos. Advances in AI give us more powerful ways to do this, let's take a look. This is a great photo. but as a parent you always want your child to be in the center of everything and it looks like the balloons were cut out in this one so you can go ahead and reposition the birthday boy magic editor which automatically recreates parts of the bench and balloons that were missing. captured in the original shot as a finishing touch, you can punch holes in the sky, change the lighting in the rest of the photo so the edit feels consistent, it's truly magical, we're excited to roll out the magic editor in Google Photos later this year , our ability to Make AI Work for Everyone depends on the continued advancement of our core models, so I want to take a moment to share how we're addressing them Today, we're set to announce our latest Palm 2 farming and production model. 2 Bill Zone are undergoing fundamental research and our latest infrastructure is highly capable in a wide range of tasks and easy to deploy.

Today we are announcing more than 25 products and features powered by Palm2. Palm 2 models offer excellent core capabilities in a wide range of sizes. We have affectionately called them order bison gecko and unicorn gecko is so lightweight that it can run on mobile devices fast enough to generate great interactive apps on the device, even offline. Palm to models are stronger in logic and reasoning thanks to extensive training in scientific and mathematical subjects. It is also trained in multilingual text, so it covers over 100 languages, so it understands and generates nuanced results, while the Palm 2 is highly capable, it really shines when domain-specific knowledge is adjusted.

We recently released Secom, a version of pump to fine-tune security use cases. Use AI to better detect malware. scripts and can help security experts understand and resolve threats. Another example is metform2. In this case, it is adjusted to medical knowledge. This adjustment achieved a 9-fold reduction in inaccurate reasoning compared to the model that approximates the performance of medical experts in answering the same set. In fact, MetForm 2 was the first language model to perform expert-level medical licensing exam-style questions and is currently state-of-the-art. We are also working on adding capabilities to metform 2 so that it can synthesize information from medical images such as airplane films and mammograms, you can imagine an AI collaborator that helps radiologists interpret images and communicating the results is the last step in our journey a decade to responsibly bring AI to billions of people.

It is based on the progress made by two world-class teams. the deep brain and deep mind team we recently brought these two teams together into a single unit

google

deepmind using

google

computing resources are focused on building more capable systems safely and responsibly this includes our next generation Gemini Foundation model which is still in Gemini training was built from the ground up to be highly efficient multimodal in tool and API integrations and built to enable future innovations like memory and scheduling, although it's still early days, we're already seeing impressive multimodal capabilities not seen in previous models once they are launched. rigorously tuned and tested for security Gemini will be available in various sizes and capacities, just like pom 2.

As we invest in more advanced models, we are also investing deeply in AI accountability, this includes having the tools to identify synthetically generated content whenever you encounter it if you look at a synthetic image. It's impressive how real it looks, so you can imagine how important it will be in the future. Metadata allows content creators to associate additional context with original files, giving you more information every time you find an image. We will make sure that each of our AI generates. images as metadata as models become better and more capable one of the most exciting opportunities is to make them available for people to interact with directly; that is the opportunity we have on board.

Large language models have captured the world's imagination by changing the way we think about the future of computing. We launched Bard as a limited access experiment in a large, lightweight language model to get feedback and iterate and the team has been ever since. working hard to make quick improvements and release them quickly with the Palm 2. Bard's mathematical logic and reasoning skills took a huge leap. Bard can now collaborate on tasks such as code generation, debugging, and explaining code snippets. Bard has already learned over 20 programming languages, including C plus plus go JavaScript, Python kotlin, and even Google Sheets functions.

Now Bart can help too. I understand the code. Could you tell me what the checkerboard does in this code? This is a very helpful explanation of what it does and makes things clearer. Well, let's see if we can improve this code a little. How could I improve it? code okay, let's see there is a list comprehension, create a function and use a generator, those are some great suggestions now, could you put them together into a single block of Python code? Well, now Bart is rebuilding the code with these improvements. Okay, great, how easy it was for us to do it!

I also heard you want the Dark theme, so starting today you can activate it directly in Bart or let it follow your OS settings in the coming weeks. Bard will become more visual in both his answers and your directions, so if you ask what some of the things you should do are, see places in New Orleans Bart will use Google search and the knowledge graph to find the most relevant images the French Quarter the Garden District these images are really giving me a much better idea of ​​what I'm exploring we'll also make it easier for you to point Bart to images that give you even more ways to explore and create imagine I'm 18 and I need apply to university.

I won't say how long it's been, but it's still an overwhelming process, so I'm thinking. about universities, but I'm not sure what I want to focus on. I like video games and what types of programs might be interesting. Okay, this is a helpful Head Start animation. It seems quite interesting. Now I could ask you to help me find universities with animation programs. in Pennsylvania, okay, great, now there's a good list of schools to see where they are. Now you could say show them on a map here. Bart will use Google Maps to visualize where the schools are.

This is very useful and it is exciting to see that there are many. of options not too far from home now let's start organizing things a little show these options as a well structured and organized table but there is more I want to know add a column that shows if they are public or private schools perfect this is a great start Go ahead and Now let's move this to Google Sheets so my family can step in later to help me with my search. You can see how easy it will be to start off embarrassed and quickly have something useful to move into apps like Docs or Sheets to develop with others, okay this is a taste of what's possible when Bard meets some of Google's apps, But that's just the beginning.

Bard will be able to access all kinds of services from across the web with extensions from amazing partners like Instacart and Khan. Academy and many more, we are eliminating the waitlist and opening Bard to more than 180 countries and territories. Bart will also be available in more languages ​​In addition to English, starting today you will be able to talk to Bart in Japanese and Korean and we are delighted. to share that we're on track to support 40 languages ​​soon, and now to hear more about how large language models are enabling next-generation productivity features right in the workspace. I will hand it over to aparna from the beginning.

The workspace was created to allow you to collaborate in real time with other people. Now you can collaborate in real time with AI. AI can act as a coach, a thinking partner, a source of inspiration and a productivity driver in all areas. workspace apps, our first steps with AI are a contributor reviews the Help Me Write feature in Gmail and documents that were released to trusted reviewers in March. One of our most popular use cases is the administrator job description that every company, large or small, needs to hire people. A good job description can make all the difference.

Here's how docs has helped you Let's say you run a fashion boutique and need to hire a textile designer to get started. Enter just a few words as a senior-level job description for textile designer. docs will accept that message and send it to our Palm 2 based model and let's see what I got, not bad, with just seven words the model came back with a good starting point written very well for me, now you can take it and customize it for the type of experience, education and skill set this role needs to save you. After a lot of time and effort, let me show you how you can get more organized with your sheets.

Imagine you have a dog walking business and you need to keep track of things like your clients' dog logistics, like what time they should be walked and how. Etc long sheets can help you organize on a new sheet. Simply type something like client and pet list for a dog walking business with rates and hit create. She sends this input to a fine-tuned model that we've been training with all sorts of specific use cases from the sheet notice that the model the model figured out what it might need the generated table has things like the dog's name information notes from the client Etc.

This is a good start for you. Playing with the leaves made it easier for him to get started, so you can get back to doing what you love. Prompts are a powerful way to collaborate with AI. The right indication can unlock much more of these models. What if AI could proactively give you even better directions? What if these prompts were actually contextual and changed based on what? you're working on my niece Mira and I are working together on a spooky story for summer camp. We have already written a few paragraphs but now we are stuck. Let's look for help.

As you can see, we launched a side panel. Something about the team. He lovingly calls his partner, his partner, instantly reads and processes the document and offers them really interesting suggestions along with an open and fast dialogue. If we look closely, we can see some of the hints, like what happened to the golden shell, what are the common mysterious plot twists, let's try the shell. option and see what comes back with now what is happeningbehind the scenes is that we have provided the full document as context for the model along with the suggested message and let's see what we got when the golden shell was eaten by a giant squid that lives in Cove, this is a good start, let's insert these as notes for so we can continue with our little project and this is exactly what AI can help with.

I see a new suggestion there for generating images, let's see what this does. The story has a town, a golden shell and other details and instead of having to write all that down, the model picks up these details from the document and generates images. Let's say you're about to give an important presentation and you've been so focused on the content that you forgot to prepare the speaker notes, the presentation is in an hour. Oh, don't panic, see what one of the suggestions is. Create speaker notes for each slide. What happened behind the scenes here is that the presentation and other relevant context were sent to the model. to help create these notes and once you've reviewed them, you can press insert and edit the notes to convey what you intended, so now you can present without worrying about the notes.

Next, we will talk about the search to provide you. an idea of ​​how we're bringing generative AI to search. I'm going to invite Kathy on stage. Let's start with the search for what is better for a family with children under three years old and a dog, Bryce Canyon or Arches, although this is the question you probably wouldn't ask this way today, you would break it down into smaller parts, examine the information and then you would put things together yourself. Now search does the heavy lifting for you. What you see here looks quite different, so let me.

First we give you a quick tour and you'll notice a new integrated search results page so you can get even more from a single search, there's an AI-powered snapshot that quickly gives you the lay of the land on a topic, so here you can see That while both parks are kid-friendly, only Bryce Canyon has more options for your furry friend. If you want to go deeper, there are links included in the snapshot, you can also click to enlarge your view and you will see how the information is corroborated so that you can consult more details and really explore the richness of the topic that builds this new experience.

Building on Google's ranking and security systems that we've been perfecting for decades, these new generative AI capabilities will make searching smarter and easier. Let's say you're looking for a good bike for a hilly five-mile ride. This can be a big buy, so you want to investigate the AI-powered snapshot, you'll see important considerations like the motor and battery to tackle those hills and the suspension for a comfortable ride below, you'll see products that fit the bill. , each with images, helpful reviews. descriptions and current prices, this is based on the Google Shopping Graph, the world's most comprehensive data set on sellers of ever-changing products.

Brand reviews and available inventory with over 35 billion listings. In fact, there are 1.8 billion live updates to our buy chart every hour and for trading purposes. For queries like this, we also know that ads can be especially useful in connecting people with useful information and helping businesses get discovered online. They are clearly labeled here and we are exploring different ways to integrate them as we roll out new experiences in search and now. If you've done some research you may want to explore further, just below the snapshot you'll see the option to ask a follow-up question or select a suggested next step.

Tapping any of these options will take you to our new conversation. So in this case you may want to follow up on electric bikes to look for one in your favorite color, red, and without having to go back to the starting point, Google search understands your entire intention and what you are specifically looking for. electric bikes in red that would be good for a five mile ride with hills and even when you're in this talk mode it's an integrated experience so you can just scroll to see other search results now maybe this electric bike seems to be A good option for your travel With just one click you will be able to see a variety of retailers that have it in stock and some that offer free delivery or returns.

You'll also see current prices, including offers, and can seamlessly go to a merchant website. Get ready to ride these new generative AI capabilities and unlock a whole new category of search experiences. It could help you create a clever name for your cycling club. Create the perfect social post to show off your new wheels or even test your knowledge of bike hand signals, these are things you may have never thought to search for before this new generative search experience, also known as sge, will be available in Labs along with a few other experiments and will be rolled out in the coming weeks.

AI is not only a powerful enabler, but it is also a big platform change. All companies and organizations are thinking about how to drive transformation. That's why we're focused on making it easy and scalable for others to innovate with AI, which means delivering the most advanced. Computing infrastructure including state-of-the-art TPUs and GPUs and expanded access to the latest Google Foundation models that have been rigorously tested on our own products. To give you more information on how we're doing this with Google Cloud, let's welcome Thomas to all the investments. those that he has heard about today are also reaching companies.

There are three ways Google Cloud can help you take advantage of the huge opportunity in front of you. First, you can build generative applications using our AI platform. Vertex AI with Vertex. You can access Foundation models for text and image chat, simply select the model you want to use, create prompts for tuning the model, and you can even tune model weights in your own dedicated compute pools to help you retrieve up-to-date, objective information from the databases of your company and your corporate Internet. your website and business applications we offer enterprise search with both Vertex and enterprise search you have sole control of your data and the costs of using generative AI models;

In other words, your data is yours and no one else's, you can also choose the best model for your specific needs in many sizes that have been optimized for cost latency and quality. The second way we help you take advantage of these opportunities is by introducing Duet AI for Google Cloud. Duet uses generative AI to support developers and can provide you with contextual code completion. offers suggestions tailored to your code base and generates complete functions in real time. It can even help you with code reviews and inspection. The third way we help you take advantage of this moment is by building all of these capabilities into our AI-optimized infrastructure. makes large-scale training workloads up to eighty percent faster and up to 50 percent cheaper compared to any available alternative.

Look when it nearly doubles the performance for less than half the cost. Incredible things happen today. We are pleased to announce a new addition to this. infrastructure family, A3 virtual machines based on the latest nvidia h100 gpus, we provide the widest range of computing options for leading AI companies, such as anthropic and mid-journey, to build their future on Google Cloud and yes, There's a lot more to come next, Josh is here to show you exactly how we're making it easy and scalable for every developer to innovate with AI and Palm 2. Thanks Thomas now for showing you how powerful the Palm API is.

I want to share a concept that five Google engineers developed. In recent weeks, the idea is called Tailwind project and we consider it as a first AI notebook that helps you learn faster, like a real notebook, your notes and your sources drive Tailwind. The way it works is that you can simply select files from Google Drive and it effectively creates a custom, private AI model that has experience with the information you give it. Now imagine that I am a student taking a computer history class. I'll open tailwind and I can quickly see all my different Notes, Tasks, and Readings in Google Drive.

I can insert them and what will happen when Tailwind loads. You can see my different notes and articles on the side. Here they are in the middle and it instantly creates a study guide on the right to guide me. I can see that you are extracting key concepts and questions based on the materials I have provided. Now I can come in here and quickly change it to go through all the different sources and write something like creating a glossary for Hopper and what's going to happen behind it. the scenes will automatically compile a glossary associated with all the different notes and articles related to Grace Hopper, the history of computing, Pioneer, check out this phlomatic compiler from Cobalt, all created based on my notes.

Now the Tailwind project is still in its infancy, but I had a lot of fun making this prototype and we realized that it's not just for students, it's useful for anyone synthesizing information from many different sources of your choosing, such as writers researching an article. or analysts making earnings calls or even lawyers preparing for a case. Imagine collaborating. with an AI That's based on what you've read in all your notes. Creating bold AI requires a responsible approach, so let me hand it over to James to share more. Thank you. Hello everyone. I'm James, in addition to research, I direct a new area.

At Google, the company's technology called generative AI makes it easier than ever to create new content, but also raises additional questions about its reliability. That's why we're developing and giving people tools to evaluate online information. In the coming months we will add two. new ways for people to evaluate images first without this image tool in Google search you will be able to see important information, such as when and where similar images may have first appeared, where else the image has been viewed online, including verification of news and social sites As we begin to implement generative image capabilities like the ones Sundar mentioned, we will ensure that each of our AI-generated images has metadata and a markup on the original file to give you context if you find it outside of our platforms, not only for creators and Publishers will be able to add similar metadata so that you can see a label on images in Google search marking them as generated by AI.

As we apply our AI principles, we also begin to see potential tensions when it comes to being bold and responsible. An example Universal Translate is an experimental AI video dubbing service that helps expert speakers translate their voice while matching their lip movements. Let me show you how it works with an online college course created in partnership with Arizona State University. What many college students don't know. is that knowing when to ask for help and then following through using helpful resources is actually a hallmark of becoming a productive adult - in universities we use next generation translation models to translate what the speaker says, models to replicate style and tone and then matching the speaker's lip movements, then we put it all together, this is a huge step forward for learning comprehension and we are seeing promising results in course completion rates, but there is a tension inherent here, you can see how this can be incredibly beneficial, but some of the same underlying bad actors could misuse the technology to create deepfakes, which is why we created the guardrail service to help prevent misuse. and make it accessible only to authorized partners.

We must conclude. I've been reflecting on the big technological shifts that we've all been part of the shift where AI is as big as it gets and that's why it's so important that we make AI useful for everyone. We approach it with courage and excitement because, as we look to the future, Google will deeply understand that insights combined with the capabilities of generative AI can transform search and all of our products. Once again, we look forward to working and building together, so on behalf of all of us at Google, thank you and enjoy the rest of IO.