Skip to main content

Building a React app using Speechly React Client

Learn how to add voice features to a React app using Speechly React Client.

Getting started​

This guide assumes you've got some basic knowledge of React app development. We'll be using a Create React App project with TypeScript. If you don't want to use TypeScript, just omit any type declarations etc.

You'll also need a Speechly account and a Speechly application that's using a Conformer model. If you are new to Speechly, you can follow our quick start guide to get started.

Installation​

Install the @speechly/react-client package.

npm install @speechly/react-client
# or
yarn add @speechly/react-client

Then, import SpeechProvider, wrap your app with the context provider and pass your App ID to it. Get your App ID from Speechly Dashboard or by using Speechly CLI list command.

src/index.tsx
import { SpeechProvider } from '@speechly/react-client';

<SpeechProvider
  appId="YOUR-APP-ID"
  debug
  logSegments
>
  <App />
</SpeechProvider>

The debug and logSegments properties might be helpful when developing, as they display changes in the client state as well as log the API output.

Next, start the development server.

npm run start
# or
yarn start

Open localhost:3000 to see the application running. If you have debug enabled, you should see from the developer console that the client has connected to the API. Now you are ready to capture microphone audio!

Capture microphone audio​

The easiest way to capture audio from the browser microphone is creating a button that toggles the microphone on and off.

First, import the useSpeechContext hook to read and subscribe to context.

src/App.tsx
import { useSpeechContext } from '@speechly/react-client';

function App() {
  const { listening, attachMicrophone, start, stop } = useSpeechContext();
  // ...

useSpeechContext is a React hook that exposes SpeechContext context, which holds the state of Speechly API client.

Next, create a button and a click handler for it where you attach the microphone to the client.

  //..
  const handleClick = async () => {
    if (listening) {
      await stop();
    } else {
      await attachMicrophone();
      await start();
    }
  };

  return (
    <div className="App">
      <button onClick={handleClick}>
        {listening ? 'Stop' : 'Start'} microphone
      </button>
    </div>
  );

attachMicrophone is a convenience method for attaching the microphone to the client. Browsers don't allow accessing the microphone programmatically, that's why it's required to call it from a user initiated action.

The start and stop methods are used for manually controlling audio processing. When used together with the listening interface, you have created simple on/off microphone toggle button.

The first time you press the microphone button you will be prompted for microphone permissions. If you have logSegments enabled, you should see the API output in the developer console.

React to API updates​

A common pattern when working with Speechly React Client is to listen for current segment change events in the useEffect hook and use useState for making changes to the app state.

First, import segment and declare two state variables for tentative and final transcripts.

src/App.tsx
import React, { useEffect, useState } from 'react';
import { useSpeechContext } from '@speechly/react-client';

function App() {
  const { segment, listening, attachMicrophone, start, stop } = useSpeechContext();
  const [transcripts, setTranscripts] = useState<string[]>([])
  const [tentative, setTentative] = useState<string>('')
  //...

Next, subscribe to changes for segment, create a transcript string from segment.words array and save it to the state variables.

  //...
  useEffect(() => {
    if (segment) {
      const transcript = segment.words.map((word) => word.value).join(' ');
      setTentative(transcript);
      if (segment.isFinal) {
        setTentative('');
        setTranscripts((current) => [...current, segment]);
      }
    }
  }, [segment]);
  //...

segment is a structure that accumulates speech recognition (ASR) and natural language understanding (NLU) results. When segment.isFinal is false, the segment might be updated several times. When true, the segment won't be updated anymore and subsequent callbacks within the same audio context refer to the next segment.

Finally, render all the tentative and final transcripts.

  //...
  return (
    <div className="App">      
      <button onClick={handleClick}>
        {listening ? 'Stop' : 'Start'} microphone
      </button>
      <div>
        {transcripts.map((transcript) => <p>{transcript}</p>)}
        <em>{tentative}</em>
     </div>
    </div>
  );
}

Next steps​

By now you should have a basic understanding of how to work with Speechly React Client and a working React app to that's able to handle speech input and and produce a transcript. Now you're ready to learn about some more advanced features of Speechly: