Gemini AI Bounding Box Demo

\n\n

This application visualizes object detection results by drawing bounding boxes on images using the Google's Gemini 1.5 Pro AI model.

\n\n

API keys are only stored in your browser's local storage.

Images are only sent to Google's Gemini API servers for processing.

\n\n

Try the following images (click the links to add them):

\n\n\n`;\n\n\nconst markdownJsonRegex = /```json\\n([\\s\\S]*?)```/;\nexport const module = `\n\n`;\n\n\n\n\nexport const html = `\n\n\n \n \n \n Gemini AI Bounding Box Demo\n \n \n \n \n \n \n \n \n \n \n \n\n ${module}\n\n \n\n \n\n\n

${intro}

\n Upload an image\n \n

\n Or enter an image URL:\n \n

\n \n\n

\n\n

Get an image description

Sometimes getting a description of the image helps you understand what the AI sees. This can help you better set up the prompt and settings.

\n \n \n

\n\n

Detect Bounding Boxes

Use this to detect bounding boxes of objects in an image. Use the image description to help with setting up the prompt. Rows and columns determine how many tiles the image will be divided into — for simple images, 1x1 is best.

For the free API you might get the \"Resource has been exhausted\" error if you make too many requests too quickly.

\n \n

\n \n Apply High Contrast Filter\n

\n Dark Threshold:\n \n

\n Light Threshold:\n \n

\n \n

\n Rows:\n \n

\n Columns:\n \n

\n Delay between requests (ms):\n \n

\n Grid Increment:\n \n

\n \n \n

Credits

This experiment uses a lot of ideas and code from Simon Willison's original post which you can read here: https://simonwillison.net/2024/Aug/26/gemini-bounding-box-visualization/

Source: https://www.val.town/v/yawnxyz/geminiBbox/
Original: https://github.com/simonw/tools/blob/main/gemini-bbox.html

\n A labspace.ai experiment \n

\n\n\n`;\n\nconst app = new Hono();\napp.get(\"/\", (c) => c.html(html));\n// Add the CORS proxy endpoint\napp.use('/proxy/*', cors());\napp.get('/proxy/*', async (c) => {\n console.log(\"[proxy request]\", c.req.url)\n const url = decodeURIComponent(c.req.url.split('/proxy/')[1]);\n if (!url) {\n return c.text('No URL provided', 400);\n }\n\n try {\n const response = await fetch(url, {\n headers: {\n 'Accept': 'image/*',\n 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36'\n }\n });\n\n if (!response.ok) {\n console.error(`Proxy error for ${url}:`, {\n status: response.status,\n statusText: response.statusText,\n headers: Object.fromEntries(response.headers.entries())\n });\n return c.text(`Failed to fetch: ${response.statusText}`, response.status);\n }\n\n const contentType = response.headers.get('content-type');\n const blob = await response.blob();\n \n return new Response(blob, {\n headers: {\n 'content-type': contentType || 'application/octet-stream',\n 'access-control-allow-origin': '*'\n }\n });\n } catch (error) {\n return c.text(`Proxy error: ${error.message}`, 500);\n }\n});\n\nexport default (typeof Deno !== \"undefined\" && Deno.env.get(\"valtown\")) ? app.fetch : app;\n\n","deleted","label",[116],"dcecbc0a-2792-11f0-9d52-569c3dd06744","README.md","dcecbd68-2792-11f0-9d52-569c3dd06744",1,["D",1724697649645],"# Gemini AI Object Detection and Bounding Box Visualizer\n\nThis application visualizes object detection results by drawing bounding boxes on images using the [Google's Gemini 1.5 Pro AI model](https://ai.google.dev/).\n\nTry using this image of 8 bananas, with 1 row and 1/2/4 columns: [https://f2.phage.directory/blogalog/bananas-8.png](https://f2.phage.directory/blogalog/bananas-8.png)\n\nOr this image of a phage plaque assay, with 3x3 grid and high contrast turned on: [https://f2.phage.directory/blogalog/pae7.png](https://f2.phage.directory/blogalog/pae7.png)\n\nThis app is an adaptation of Simon Willison's idea, which you can read more about here: [https://simonwillison.net/2024/Aug/26/gemini-bounding-box-visualization/](https://simonwillison.net/2024/Aug/26/gemini-bounding-box-visualization/)\n\n- API keys are only stored in your browser's local storage.\n- Source code: [https://www.val.town/v/yawnxyz/geminiBbox/](https://www.val.town/v/yawnxyz/geminiBbox/)\n- Original Source: https://github.com/simonw/tools/blob/main/gemini-bbox.html\n\nMigrated from folder: Apps/GeminiBbox/geminiBbox",[132],"pulls","forks","envVars",{"_144":52,"_146":52},"connections",{"_144":52,"_145":52},"count","unfilledCount","emptyCount",{"_20":131,"_13":132,"_117":133,"_118":-5,"_119":21,"_120":134,"_121":38,"_22":153,"_29":52,"_124":101,"_126":136,"_128":75,"_129":-5,"_150":154,"_89":155},{"_20":115,"_13":116,"_117":47,"_118":-5,"_119":21,"_120":30,"_121":38,"_22":149,"_29":123,"_124":125,"_126":127,"_128":75,"_129":-5,"_150":151,"_89":152},["D",1730409719115],"children",[],[116],["D",1724697649645],[],[132],["D",1746217217347],{"_11":12,"_87":83},{"_11":12,"_81":-5,"_82":83,"_44":45,"_84":85,"_86":75},{"_29":30,"_13":26,"_20":21,"_27":28,"_31":-5,"_32":-5,"_76":-5,"_77":-5,"_78":-5,"_79":14,"_80":12},"ctx",{"_68":-5,"_109":193,"_194":195,"_196":197,"_198":75,"_199":75,"_200":75,"_201":38,"_202":-5,"_203":-7,"_204":-7,"_205":52,"_206":75,"_207":75,"_208":-5,"_209":75,"_210":75,"_211":75,"_212":-5},"ENV",{"_164":165,"_166":167,"_168":26,"_169":170,"_171":172,"_173":174,"_175":176,"_177":178,"_179":180,"_181":182,"_183":184,"_185":186,"_187":188,"_189":190,"_191":192},"REACT_APP_SENTRY_DSN","https://586020d86f05489db63a554690acf1e9@o1398359.ingest.sentry.io/6724499","RENDER_GIT_COMMIT","e87eb04e707cf64e7e532c8e8ec395f209e17f15","RENDER_GIT_BRANCH","POSTHOG_API_TOKEN","phc_nmoyuAkStEZuUp69rYlg1DZgrIt19Gx0Y4Hwf8tGgJP","POSTHOG_API_HOST","https://wart.val.town","WEB_API_TEMPLATE","https://{val}.web.val.run","APP_URL","https://www.val.town","MODULE_URL","https://esm.town","VANITY_WEB_API_TEMPLATE","https://{label}.val.run/","DENOLS_URL","https://lsp.val.town","NODE_ENV","production","CLOUDFLARE_TURNSTILE_SITE_KEY","0x4AAAAAACtaIZJH7s9rBMZ6","GOOGLE_CLIENT_ID","875989696899-jtsqkhlbdv5jrlsk8631p7ipjg3c6o62.apps.googleusercontent.com","STRIPE_PUBLISHABLE_KEY","pk_live_51N2zvWEpZTTLipIkVEwXrUG9AT0yeOOEYxszarZT1jpsZV58Ly6wlg3SltZpHN26CUalOBmUSY7adQRQiSkCZz2500qSXNkf8Y","API_URL","https://api.val.town",[],"preferencesKeybindings","default","preferencesTheme","light","preferencesFixers","preferencesCopilot","preferencesSelectionAi","preferencesCloseBrackets","preferencesChatPanelState","preferencesDprint","preferencesDenofmt","notificationCount","flagDenols","isBusinessOrgMember","businessOrgHandle","isBusinessOrgAdmin","isHigherTownieTier","navCollapsed","currentOrgId"]

Branches1

Branches
1