Getting started

How to install and use Speechly Client Libraries

Note that you also need a Speechly application id. This you can obtain from the Dashboard for one of the ready-made example applications, or after having configured one yourself. And don’t forget to deploy your application on the Dashboard as well!

Install and import the client library

Speechly Web Components help you get started quickly. Push-To-Talk Button component integrates browser-client library (which you can of course install via npm as well) and requires no other dependencies to work. Just add the resources in your head block:

<head>
  <script type="text/javascript" src="https://unpkg.com/@speechly/browser-ui/core/push-to-talk-button.js"></script>
  <script type="text/javascript" src="https://unpkg.com/@speechly/browser-ui/core/big-transcript.js"></script>
  <script type="text/javascript" src="https://unpkg.com/@speechly/browser-ui/core/error-panel.js"></script>
</head>
Install the client with npm:
npm install --save @speechly/react-client
npm install --save @speechly/react-ui
And import the components into your application:
import React from "react";
import { SpeechProvider, useSpeechContext } from "@speechly/react-client";
The Speechly iOS client is distributed using Swift Package Manager, add it as a dependency to your Package.swift:
// swift-tools-version:5.3
import PackageDescription

let package = Package(
    name: "MySpeechlyApp",
    dependencies: [
        .package(name: "speechly-ios-client", url: "https://github.com/speechly/ios-client.git", from: "0.3.0"),
    ],
    targets: [
        .target(name: "MySpeechlyApp", dependencies: []),
        .testTarget(name: "MySpeechlyAppTests", dependencies: ["MySpeechlyApp"]),
    ]
)

Import the client…

import Speechly

… and instantiate it in your manager class:

let client: Speechly.Client

public init() {
    client = try! Speechly.Client(
        appId: UUID(uuidString: "YOUR_APP_ID_HERE")!,
    )
    client.delegate = self
    ...
}

The client package is available from our Github repository:
https://github.com/speechly/android-client/releases/latest.

Add android-client to your build.gradle dependencies.
dependencies {
  implementation 'com.speechly:android-client:latest'
}

Import the client, as well as the SpeechlyButton.

import com.speechly.client.speech.Client
import com.speechly.client.slu.*
import com.speechly.ui.SpeechlyButton

Instantiate the client in your main activity and add a SpeechlyButton. Make sure to replace YOUR_APP_ID_HERE with the correct Speecly app id.

private val speechlyClient: Client = Client.fromActivity(
        activity = this,
        UUID.fromString("YOUR_APP_ID_HERE")
)

private var button: SpeechlyButton? = null

Take UI components to use

Ready-made UI components provide a Push-To-Talk Button for starting and stopping listening for speech, and a display component for showing the returned transcript (not yet available on Android). It is not necessary to use these, but we highly recommend this as a first step to get started quickly.

Include the following lines in your body:
<big-transcript
  placement="top" >
</big-transcript>

<push-to-talk-button
  appid="YOUR_APP_ID_FROM_SPEECHLY_DASHBOARD"
  placement="bottom" >
</push-to-talk-button>

<error-panel
  placement="bottom" >
</error-panel>
Import the components:
import {
  PushToTalkButton,
  BigTranscript,
  ErrorPanel
} from "@speechly/react-ui";
Place the components inside your block
function App() {
  return (
    <SpeechProvider appId="014ce3a6-9bbf-4605-976f-087a8f3ec178" language="en-US">
      <BigTranscript placement="top"/>
      <PushToTalkButton placement="bottom" captureKey=" "/>
      <ErrorPanel placement="bottom"/>
    </SpeechProvider>
  );
}
Initialise the TranscriptView and MicrophoneButtonView, and add them in the addViews function of your manager class.
let transcriptView = TranscriptView()
lazy var speechButton = MicrophoneButtonView(delegate: self)

public func addViews(view: UIView) {
        view.addSubview(transcriptView)
        view.addSubview(speechButton)
        ...
}

Implement an extension that maps the microphone button to starting and stopping the client.

extension SpeechlyManager: MicrophoneButtonDelegate {
    func didOpenMicrophone(_ button: MicrophoneButtonView) {
        self.client.startContext()
    }
    func didCloseMicrophone(_ button: MicrophoneButtonView) {
        self.client.stopContext()
    }
}

In your main activity, define a touch listener that activates and deactivates audio transmission. Also, in the onCreate method assign this listener to SpeechlyButton.
private var buttonTouchListener = object : View.OnTouchListener {
    override fun onTouch(v: View?, event: MotionEvent?): Boolean {
        when (event?.action) {
            MotionEvent.ACTION_DOWN -> {
                speechlyClient.startContext()
            }
            MotionEvent.ACTION_UP -> {
                speechlyClient.stopContext()
            }
        }
        return true
    }
}

override fun onCreate(savedInstanceState: Bundle?) {
    ...
    this.button = findViewById(R.id.speechly)
    this.button?.setOnTouchListener(buttonTouchListener)
    ...
}

Handle speech input

The basic idea of the client-side API is to provide a “live view” to the ongoing speech segment. This is done by registering a callback function that receives a Segment data structure as argument. This data structure represents the ongoing speech segment. Also see our Client API reference.

After an audio context has started, callback is called every time the Speechly API returns an event that updates the currently ongoing segment.

Assign a listener to the push-to-talk-button that handles broadcasted updates to the ongoing SpeechSegment.
<script type="text/javascript">
document
  .getElementsByTagName("push-to-talk-button")[0]
  .addEventListener("speechsegment", (e) => {
    const segment = e.detail;

    // Handle speech segment and make tentative changes to app state
    console.log("speechsegment message:", segment);

    if (segment.isFinal) {
      // Handle speech segment and make permanent changes to app state
      // Optionally show confirmation
      window.postMessage({ type: "speechhandled", success: true }, "*");
    }
 });
</script>
function SpeechlyApp() {
  const { segment } = useSpeechContext()
  return (
    <div>
      {segment ? <div className="segment">{segment.words.map(w => w.value).join(' ')}</div> : null}
    </div>
  )
}
Implement the `Speechly.SpeechlyDelegate` for reacting to recognition results.
extension SpeechlyManager: SpeechlyDelegate {
    func speechlyClientDidUpdateSegment(_ client: SpeechlyProtocol, segment: Segment) {
        // segment handling logic goes here
        ...
        // If you are using the TranscriptView component, also add
        DispatchQueue.main.async {
            self.transcriptView.configure(segment: segment, animated: true)
        }
    }
}
In the onCreate method, define the callback to be used for handling a Segment.
override fun onCreate(savedInstanceState: Bundle?) {
    ...
    GlobalScope.launch(Dispatchers.Default) {
        speechlyClient.onSegmentChange { segment: Segment ->
            // segment handling logic goes here
            ...
        }
    }
    ...
}
For more details, please see our Client API reference, as well as the source code of Segment on Github.

Learn more!

Check out our examples and demos on GitHub.

Also, learn how to configure your own application.


Profile image for Hannes Heikinheimo

Last updated by Hannes Heikinheimo on November 22, 2021 at 17:04 +0200

Found an error on our documentation? Please file an issue or make a pull request