Fix missing api_utils module and mitigate dependency confusion risk in HeAR demo

.dockerignore

@@ -0,0 +1,4 @@
+# Exclude files that are not needed in the serving image from the Docker build
+# context.
+
+/python/data_processing/audio_utils.py

notebooks/README.md

@@ -0,0 +1,23 @@
+# HeAR Notebooks
+
+*   Quick start with Hugging Face -
+    Example of encoding a health-related audio clip into an
+    embedding vector by running the model locally from Hugging Face. There are
+    different versions of this notebook for the following model formats:
+
+    * [TensorFlow](quick_start_with_hugging_face.ipynb)
+    * [PyTorch](quick_start_with_hugging_face_pytorch.ipynb)
+
+*   [Quick start with Vertex Model Garden](quick_start_with_model_garden.ipynb) -
+    Example of serving the model on
+    [Vertex AI](https://cloud.google.com/vertex-ai/docs/predictions/overview)
+    and using Vertex AI APIs to encode health-related audio to
+    embeddings in online or batch workflows.
+
+*   [Train a data efficient classifier](train_data_efficient_classifier.ipynb) -
+    Example of using the generated embeddings to train a custom classifier with
+    less data and compute.
+
+*   [Health event detector demo](hear_event_detector_demo.ipynb) - Example of
+    using HeAR with the health event detector models to first identify and then
+    generate embeddings for health-related sounds like coughing and breathing.

notebooks/quick_start_with_hugging_face.ipynb

@@ -32,12 +32,12 @@
         "  \u003c/td\u003e  \n",
         "  \u003ctd style=\"text-align: center\"\u003e\n",
         "    \u003ca href=\"https://github.com/google-health/hear/blob/master/notebooks/quick_start_with_hugging_face.ipynb\"\u003e\n",
-        "      \u003cimg alt=\"GitHub logo\" src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" width=\"32px\"\u003e\u003cbr\u003e View on GitHub\n",
+        "      \u003cimg alt=\"GitHub logo\" src=\"https://github.githubassets.com/assets/GitHub-Mark-ea2971cee799.png\" width=\"32px\"\u003e\u003cbr\u003e View on GitHub\n",
         "    \u003c/a\u003e\n",
         "  \u003c/td\u003e\n",
         "  \u003ctd style=\"text-align: center\"\u003e\n",
         "    \u003ca href=\"https://huggingface.co/google/hear\"\u003e\n",
-        "      \u003cimg alt=\"HuggingFace logo\" src=\"https://huggingface.co/front/assets/huggingface_logo-noborder.svg\" width=\"32px\"\u003e\u003cbr\u003e View on HuggingFace\n",
+        "      \u003cimg alt=\"Hugging Face logo\" src=\"https://huggingface.co/front/assets/huggingface_logo-noborder.svg\" width=\"32px\"\u003e\u003cbr\u003e View on Hugging Face\n",
         "    \u003c/a\u003e\n",
         "  \u003c/td\u003e\n",
         "\u003c/tr\u003e\u003c/tbody\u003e\u003c/table\u003e\n",

notebooks/quick_start_with_hugging_face_pytorch.ipynb

@@ -0,0 +1,264 @@
+{
+  "cells": [
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "1fgVWTMK9SNz"
+      },
+      "source": [
+        "~~~\n",
+        "Copyright 2025 Google LLC\n",
+        "\n",
+        "Licensed under the Apache License, Version 2.0 (the \"License\");\n",
+        "you may not use this file except in compliance with the License.\n",
+        "You may obtain a copy of the License at\n",
+        "\n",
+        "    https://www.apache.org/licenses/LICENSE-2.0\n",
+        "\n",
+        "Unless required by applicable law or agreed to in writing, software\n",
+        "distributed under the License is distributed on an \"AS IS\" BASIS,\n",
+        "WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
+        "See the License for the specific language governing permissions and\n",
+        "limitations under the License.\n",
+        "~~~\n",
+        "\n",
+        "\n",
+        "# Quick start with Hugging Face (PyTorch model)\n",
+        "\n",
+        "\u003ctable\u003e\u003ctbody\u003e\u003ctr\u003e\n",
+        "  \u003ctd style=\"text-align: center\"\u003e\n",
+        "    \u003ca href=\"https://colab.research.google.com/github/google-health/hear/blob/master/notebooks/quick_start_with_hugging_face_pytorch.ipynb\"\u003e\n",
+        "      \u003cimg alt=\"Google Colab logo\" src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" width=\"32px\"\u003e\u003cbr\u003e Run in Google Colab\n",
+        "    \u003c/a\u003e\n",
+        "  \u003c/td\u003e  \n",
+        "  \u003ctd style=\"text-align: center\"\u003e\n",
+        "    \u003ca href=\"https://github.com/google-health/hear/blob/master/notebooks/quick_start_with_hugging_face_pytorch.ipynb\"\u003e\n",
+        "      \u003cimg alt=\"GitHub logo\" src=\"https://github.githubassets.com/assets/GitHub-Mark-ea2971cee799.png\" width=\"32px\"\u003e\u003cbr\u003e View on GitHub\n",
+        "    \u003c/a\u003e\n",
+        "  \u003c/td\u003e\n",
+        "  \u003ctd style=\"text-align: center\"\u003e\n",
+        "    \u003ca href=\"https://huggingface.co/google/hear-pytorch\"\u003e\n",
+        "      \u003cimg alt=\"Hugging Face logo\" src=\"https://huggingface.co/front/assets/huggingface_logo-noborder.svg\" width=\"32px\"\u003e\u003cbr\u003e View on Hugging Face\n",
+        "    \u003c/a\u003e\n",
+        "  \u003c/td\u003e\n",
+        "\u003c/tr\u003e\u003c/tbody\u003e\u003c/table\u003e\n",
+        "\n",
+        "This Colab notebook provides a basic usage example of the HeAR encoder that generates a machine learning representation (known as \"embeddings\") from health-related sounds (2-second audio clips sampled at 16kHz). These embeddings can be used to develop custom machine learning models for health acoustic use-cases with less data and compute compared to traditional model development methods.\n",
+        "\n",
+        " Learn more about embeddings and their benefits at [this page](https://developers.google.com/health-ai-developer-foundations/hear)."
+      ]
+    },
+    {
+      "metadata": {
+        "id": "EvQJsuAZ50-Z"
+      },
+      "cell_type": "markdown",
+      "source": [
+        "## Install dependencies"
+      ]
+    },
+    {
+      "metadata": {
+        "id": "qdrXvhfo6dMV"
+      },
+      "cell_type": "code",
+      "source": [
+        "! git clone https://github.com/Google-Health/hear.git\n",
+        "! pip install --upgrade --quiet transformers==4.50.3"
+      ],
+      "outputs": [],
+      "execution_count": null
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "_uZFXCSuqr1V"
+      },
+      "source": [
+        "## Authenticate with HuggingFace, skip if you have a HF_TOKEN secret"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "9XpCla68-Iol"
+      },
+      "outputs": [],
+      "source": [
+        "from huggingface_hub.utils import HfFolder\n",
+        "\n",
+        "if HfFolder.get_token() is None:\n",
+        "    from huggingface_hub import notebook_login\n",
+        "    notebook_login()"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "SWckxRXiqc3L"
+      },
+      "source": [
+        "## Load and play cough audio recording"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "z2vEauhtvTan"
+      },
+      "outputs": [],
+      "source": [
+        "SAMPLE_RATE = 16000  # Samples per second (Hz)\n",
+        "CLIP_DURATION = 2    # Duration of the audio clip in seconds\n",
+        "CLIP_LENGTH = SAMPLE_RATE * CLIP_DURATION  # Total number of samples"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "h1Lw92M1b7Dd"
+      },
+      "outputs": [],
+      "source": [
+        "!wget -nc https://upload.wikimedia.org/wikipedia/commons/b/be/Woman_coughing_three_times.wav"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "5W7BMKk9cerB"
+      },
+      "outputs": [],
+      "source": [
+        "from scipy.io import wavfile\n",
+        "\n",
+        "# Load file\n",
+        "with open('Woman_coughing_three_times.wav', 'rb') as f:\n",
+        "  original_sampling_rate, audio_array = wavfile.read(f)\n",
+        "\n",
+        "print(f\"Sample Rate: {original_sampling_rate} Hz\")\n",
+        "print(f\"Data Shape: {audio_array.shape}\")\n",
+        "print(f\"Data Type: {audio_array.dtype}\")\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "1l-eFOh0mq1r"
+      },
+      "outputs": [],
+      "source": [
+        "from IPython.display import Audio, display\n",
+        "import importlib\n",
+        "audio_utils = importlib.import_module(\n",
+        "    \"hear.python.data_processing.audio_utils\"\n",
+        ")\n",
+        "resample_audio_and_convert_to_mono = audio_utils.resample_audio_and_convert_to_mono\n",
+        "\n",
+        "\n",
+        "audio_array = resample_audio_and_convert_to_mono(\n",
+        "  audio_array=audio_array, \n",
+        "  sampling_rate=original_sampling_rate,\n",
+        "  new_sampling_rate=SAMPLE_RATE,\n",
+        ")\n",
+        "display(Audio(audio_array, rate=SAMPLE_RATE))"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "fJIdGmpFqhi8"
+      },
+      "source": [
+        "## Compute embeddings"
+      ]
+    },
+    {
+      "metadata": {
+        "id": "MIRuwbfz2a-l"
+      },
+      "cell_type": "code",
+      "source": [
+        "from transformers import AutoModel\n",
+        "\n",
+        "\n",
+        "# Load the model directly from Hugging Face Hub\n",
+        "loaded_model = AutoModel.from_pretrained(\"google/hear-pytorch\")"
+      ],
+      "outputs": [],
+      "execution_count": null
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "Fhk4GKkwsAMj"
+      },
+      "outputs": [],
+      "source": [
+        "import matplotlib.pyplot as plt\n",
+        "import numpy as np\n",
+        "import torch\n",
+        "\n",
+        "preprocess_audio = audio_utils.preprocess_audio\n",
+        "\n",
+        "# This index corresponds to a cough and was determined by hand. In practice, you\n",
+        "# would need a detector.\n",
+        "START = 0\n",
+        "\n",
+        "# Add batch dimension\n",
+        "input_tensor = np.expand_dims(audio_array[START: START + CLIP_LENGTH], axis=0)\n",
+        "\n",
+        "# Call inference\n",
+        "infer = lambda audio_array: loaded_model.forward(\n",
+        "    preprocess_audio(audio_array), return_dict=True, output_hidden_states=True)\n",
+        "output = infer(torch.Tensor(input_tensor))\n",
+        "\n",
+        "# Extract the embedding vector\n",
+        "embedding_vector = np.asarray(output.pooler_output.detach()).flatten()\n",
+        "print(\"Size of embedding vector:\", len(embedding_vector))\n",
+        "\n",
+        "# Plot the embedding vector\n",
+        "plt.figure(figsize=(12, 4))\n",
+        "plt.plot(embedding_vector)\n",
+        "plt.title('Embedding Vector')\n",
+        "plt.xlabel('Index')\n",
+        "plt.ylabel('Value')\n",
+        "plt.grid(True)\n",
+        "plt.show()"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "PHTxQttKYNpa"
+      },
+      "source": [
+        "# Next steps\n",
+        "\n",
+        "Explore the other [notebooks](https://github.com/google-health/hear/blob/master/notebooks) to learn what else you can do with the model."
+      ]
+    }
+  ],
+  "metadata": {
+    "colab": {
+      "name": "quick_start_with_hugging_face_pytorch.ipynb",
+      "private_outputs": true,
+      "provenance": []
+    },
+    "kernelspec": {
+      "display_name": "Python 3",
+      "name": "python3"
+    },
+    "language_info": {
+      "name": "python"
+    }
+  },
+  "nbformat": 4,
+  "nbformat_minor": 0
+}

notebooks/quick_start_with_model_garden.ipynb

@@ -40,7 +40,7 @@
         "  </td>\n",
         "  <td style=\"text-align: center\">\n",
         "    <a href=\"https://github.com/google-health/hear/blob/master/notebooks/quick_start_with_model_garden.ipynb\">\n",
-        "      <img alt=\"GitHub logo\" src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" width=\"32px\"><br> View on GitHub\n",
+        "      <img alt=\"GitHub logo\" src=\"https://github.githubassets.com/assets/GitHub-Mark-ea2971cee799.png\" width=\"32px\"><br> View on GitHub\n",
         "    </a>\n",
         "  </td>\n",
         "</tr></tbody></table>"

notebooks/train_data_efficient_classifier.ipynb

@@ -32,12 +32,12 @@
         "  \u003c/td\u003e  \n",
         "  \u003ctd style=\"text-align: center\"\u003e\n",
         "    \u003ca href=\"https://github.com/google-health/hear/blob/master/notebooks/train_data_efficient_classifier.ipynb\"\u003e\n",
-        "      \u003cimg alt=\"GitHub logo\" src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" width=\"32px\"\u003e\u003cbr\u003e View on GitHub\n",
+        "      \u003cimg alt=\"GitHub logo\" src=\"https://github.githubassets.com/assets/GitHub-Mark-ea2971cee799.png\" width=\"32px\"\u003e\u003cbr\u003e View on GitHub\n",
         "    \u003c/a\u003e\n",
         "  \u003c/td\u003e\n",
         "  \u003ctd style=\"text-align: center\"\u003e\n",
         "    \u003ca href=\"https://huggingface.co/google/hear\"\u003e\n",
-        "      \u003cimg alt=\"HuggingFace logo\" src=\"https://huggingface.co/front/assets/huggingface_logo-noborder.svg\" width=\"32px\"\u003e\u003cbr\u003e View on HuggingFace\n",
+        "      \u003cimg alt=\"Hugging Face logo\" src=\"https://huggingface.co/front/assets/huggingface_logo-noborder.svg\" width=\"32px\"\u003e\u003cbr\u003e View on Hugging Face\n",
         "    \u003c/a\u003e\n",
         "  \u003c/td\u003e\n",
         "\u003c/tr\u003e\u003c/tbody\u003e\u003c/table\u003e\n",