Whisper Spoken LID (v2.0)

About this version

  • Submitter: keighrim
  • Submission Time: 2026-11-11T15:14:58+00:00
  • Prebuilt Container Image: ghcr.io/clamsproject/app-whisper-spoken-lid:v2.0

  • Release Notes

    Changes in parameter, can run in custom label model

    • mergeWindow: renamed to tfMerge
    • labelMap: restric output labels to certain language (can be used to turn the app into binary ot ternary classifier)
    • minLabelScore: when using a smaller set of labelMap, adjust minimal score to be “positive” (default is 0.5 which is always absolute majority under binary labels)

About this app (See raw metadata.json)

Spoken language identification over fixed-length windows of an audio track, using the language detection of OpenAI Whisper. Consecutive windows with the same language can be merged into one TimeFrame.

Inputs

(Note: “*” as a property value means that the property is required but can be any value.)

One of the following is required: [

]

Configurable Parameters

(Note: Multivalued means the parameter can have one or more values.)

  • model: optional, defaults to tiny

    • Type: string
    • Multivalued: False
    • Choices: tiny, base, small, medium, large, turbo

    Whisper model size.

  • windowMs: optional, defaults to 30000

    • Type: integer
    • Multivalued: False

    Window length in milliseconds, at most 30000.

  • minLastWindowMs: optional, defaults to 0

    • Type: integer
    • Multivalued: False

    Shortest final window to keep, in milliseconds. 0 keeps any final window.

  • tfMerge: optional, defaults to true

    • Type: boolean
    • Multivalued: False
    • Choices: false, true

    Join consecutive windows with the same label into one TimeFrame. When false, each window becomes its own TimeFrame, with one score.

  • labelMap: optional, defaults to []

    • Type: string
    • Multivalued: True

    Labels to keep, as “IN:OUT” (rename IN to OUT) or “IN” (keep IN as it is). Other labels become “-“. Empty keeps every label of the model. See README.md in the app repository for how the label is chosen.

  • minLabelScore: optional, defaults to 0.5

    • Type: number
    • Multivalued: False

    Minimum score for a label from labelMap to be kept; a window whose best mapped label scores less becomes “-“. Only applies with labelMap.

  • batchSize: optional, defaults to 1

    • Type: integer
    • Multivalued: False

    Number of windows passed to the model at once.

  • pretty: optional, defaults to false

    • Type: boolean
    • Multivalued: False
    • Choices: false, true

    The JSON body of the HTTP response will be re-formatted with 2-space indentation

  • runningTime: optional, defaults to true

    • Type: boolean
    • Multivalued: False
    • Choices: false, true

    The running time of the app will be recorded in the view metadata

  • hwFetch: optional, defaults to false

    • Type: boolean
    • Multivalued: False
    • Choices: false, true

    The hardware information (architecture, GPU and vRAM) will be recorded in the view metadata

  • tfSamplingMode: optional, defaults to representatives

    • Type: string
    • Multivalued: False
    • Choices: representatives, single, all

    Sampling mode for TimeFrame annotations. Has no effect when the app does not process TimeFrames. “representatives” uses all representative timepoints if present, otherwise skips the TimeFrame. “single” uses the middle representative if present, otherwise extracts an image from the midpoint of the start/end interval (midpoint is calculated by floor division of the sum of start and end). “all” uses all target timepoints if present, otherwise extracts all images from the time interval.

Outputs

(Note: “*” as a property value means that the property is required but can be any value.)

(Note: Not all output annotations are always generated.)

  • http://clams.ai/vocabulary/type/TimeFrame/v6
    • timeUnit = “milliseconds”
    • document = “*”
    • labelset = “*”
    • label = “*”
    • scores = “*”

    Each TimeFrame covers consecutive windows that Whisper labels with the language label. Language labels are canonicalized to BCP 47 tags, and labelMap can reduce them; see README.md in the app repository for details. labelset lists all labels the view can have. scores lists, in time order, the Whisper probability of label in each of those windows, one float in [0, 1] per window. Window i starts at start + i * windowMs; the last window ends at end and can be shorter. The probabilities are not calibrated.