Create your own video conference web application using Java & JavaScript - Its All Binary

In this article, we will create a simple video conferencing application using Java & pure JavaScript (WebRTC).

Video Conferencing Application in this article

We will create simple 2 person video conferencing application.
First user will join video conference from a desktop. His own audio/video will start showing up on browser.
Other participant will also join video conference from a android mobile device.
Then both participants will be able to see & hear each other via video conference.
Participants will have option to leave video conference using a provided button.

Technologies

For this application, we will be using below technologies

WebRTC (Web Real-Time Communication) using HTML5 & Javascript (API specification on Mozilla)
Websocket using Javascript & Java server for signaling. (Learn Websocket with simple example here)

Here are few ideas/technologies that were discarded in favor of above technologies

Desktop application client – For this, we need Java based API to connect to web cam & audio etc. There are Java provided API like JMF (Java Media Framework) etc. which have been without update from long time. There are other alternatives (like bytedeco javacv, QuickTime for Java & more) which were not considered/explored as goal was to create basic example without external libraries.
Web Application with Java server to transmit audio/video between participant browsers – Javascript can do job of capturing webcam/audio & send to server in chunks. We can use web socket server endpoint to exchange audio & video between participants in chunks. But this approach won’t be real time as server will add lag due to latency, processing time etc.

Hence finally WebRTC is chosen for this article since it provides real time Peer-To-Peer streaming between web pages. Basically with this approach, your browser will directly stream audio & video to other participant’s browser without any server in between. Server will only do job of connecting both participant’s browsers to each other.

Design

This video explains the design & approach we will be taking for this example.

Lets Code

Now that you know the design approach, lets get coding. We will create our own signaling server endpoint using Java websocket API. Below is the Java Websocket server endpoint code.

package com.itsallbinary.tutorial.vconf;

import java.io.IOException;
import java.util.Collections;
import java.util.HashSet;
import java.util.Set;

import javax.websocket.EncodeException;
import javax.websocket.OnClose;
import javax.websocket.OnMessage;
import javax.websocket.OnOpen;
import javax.websocket.Session;
import javax.websocket.server.ServerEndpoint;

/**
 * Signaling server to WebRTC video conferencing.
 */
@ServerEndpoint("/signal")
public class WebRtcSignalingEndpoint {

	private static final Set<Session> sessions = Collections.synchronizedSet(new HashSet<Session>());

	@OnOpen
	public void whenOpening(Session session) throws IOException, EncodeException {
		System.out.println("Open!");
		// Add websocket session to a global set to use in OnMessage.
		sessions.add(session);
	}

	@OnMessage
	public void process(String data, Session session) throws IOException {
		System.out.println("Got signal - " + data);
		/*
		 * When signal is received, send it to other participants other than self. In
		 * real world, signal should be sent to only participant's who belong to
		 * particular video conference.
		 */
		for (Session sess : sessions) {
			if (!sess.equals(session)) {
				sess.getBasicRemote().sendText(data);
			}
		}
	}

	@OnClose
	public void whenClosing(Session session) {
		System.out.println("Close!");
		sessions.remove(session);
	}
}

package com.itsallbinary.tutorial.vconf;

import java.io.IOException;

import java.util.Collections;

import java.util.HashSet;

import java.util.Set;

import javax.websocket.EncodeException;

import javax.websocket.OnClose;

import javax.websocket.OnMessage;

import javax.websocket.OnOpen;

import javax.websocket.Session;

import javax.websocket.server.ServerEndpoint;

/**

* Signaling server to WebRTC video conferencing.

@ServerEndpoint("/signal")

public class WebRtcSignalingEndpoint {

private static final Set<Session> sessions = Collections.synchronizedSet(new HashSet<Session>());

@OnOpen

public void whenOpening(Session session) throws IOException, EncodeException {

System.out.println("Open!");

// Add websocket session to a global set to use in OnMessage.

sessions.add(session);

}

@OnMessage

public void process(String data, Session session) throws IOException {

System.out.println("Got signal - " + data);

* When signal is received, send it to other participants other than self. In

* real world, signal should be sent to only participant's who belong to

* particular video conference.

for (Session sess : sessions) {

if (!sess.equals(session)) {

sess.getBasicRemote().sendText(data);

}

@OnClose

public void whenClosing(Session session) {

System.out.println("Close!");

sessions.remove(session);

}

This is the simple HTML where both participant’s video will be rendered. You can have another separate HTML with “Join Conference” link which can simply redirect to below HTML.

<!DOCTYPE html>
<html>
<head>
<meta name="viewport" content="width=device-width,initial-scale=1,maximum-scale=1" />
<title>WebRTC Video Conferencing Application</title>
</head>
<body>
	<!-- Title & header of demo aplication. -->
	<div>
		<img style="float: left; width: auto; height: 50px"
			src="https://itsallbinary.com/wp-content/uploads/2017/03/final_itsallbinary.gif" />
		<h3 style="position: relative; left: 10px;">WebRTC Video Conferencing <br />Application Demo
		</h3>
	</div>

	<!-- Other person's camera video will show up here -->
	<div>
		<h3 style="margin: 5px">Other Person</h3>
		<video style="width: 50vh; height: 50vh;" id="remoteVideo"
			poster="https://img.icons8.com/fluent/48/000000/person-male.png" autoplay></video>
	</div>

	<!-- Your camera video will show up here. -->
	<div>
		<h3 style="margin: 5px">You</h3>
		<video style="width: auto; height: 20vh;" id="localVideo"
			poster="https://img.icons8.com/fluent/48/000000/person-male.png" autoplay muted></video>
	</div>

	<!-- Button to leave video conference. -->
	<div class="box">
		<button id="leaveButton" style="background-color: #008CBA; color: white; ">Leave Video Conference</button>
	</div>

	<script type="text/javascript" src="conf.js?reload=true"></script>

</body>
</html>

<!DOCTYPE html>

<html>

<head>

<title>WebRTC Video Conferencing Application</title>

</head>

<body>

<div>

<img style="float: left; width: auto; height: 50px"

src="https://itsallbinary.com/wp-content/uploads/2017/03/final_itsallbinary.gif" />

<h3 style="position: relative; left: 10px;">WebRTC Video Conferencing <br />Application Demo

</h3>

</div>

<div>

<h3 style="margin: 5px">Other Person</h3>

<video style="width: 50vh; height: 50vh;" id="remoteVideo"

poster="https://img.icons8.com/fluent/48/000000/person-male.png" autoplay></video>

</div>

<div>

<video style="width: auto; height: 20vh;" id="localVideo"

poster="https://img.icons8.com/fluent/48/000000/person-male.png" autoplay muted></video>

</div>

<button id="leaveButton" style="background-color: #008CBA; color: white; ">Leave Video Conference</button>

</div>

</body>

</html>

Now comes the main part i.e. JavaScript. Notice that we are using Google’s free public STUN server for this example. Even though this looks like a big script, have made it pretty simple with code comments explaining all steps. This JavaScript does below steps,

On load, prepares websocket connection for signaling server endpoint.
Captures media devices i.e. webcam & microphone using navigator.mediaDevices then renders in UI as local video.
Performs offer-answer handshake using Java Signaling websocket server & Google’s STUN server so that all participants get connectivity information of each other.
Setup peer connection with other participant.
Then adds local video & audio stream as track to peer connection so other participant can hear & see you.

var peerConnection;

/*
 * Setup 'leaveButton' button function.
 */
const leaveButton = document.getElementById('leaveButton');
leaveButton.addEventListener('click', leave);

function leave() {
    console.log('Ending call');
    peerConnection.close();
    signalingWebsocket.close();
    window.location.href = './index.html';
};

/*
 * Prepare websocket for signaling server endpoint.
 */
var signalingWebsocket = new WebSocket("ws://" + window.location.host +
    "/video-conf-tutorial/signal");

signalingWebsocket.onmessage = function(msg) {
    console.log("Got message", msg.data);
    var signal = JSON.parse(msg.data);
    switch (signal.type) {
        case "offer":
            handleOffer(signal);
            break;
        case "answer":
            handleAnswer(signal);
            break;
        // In local network, ICE candidates might not be generated.
        case "candidate":
            handleCandidate(signal);
            break;
        default:
            break;
    }
};

signalingWebsocket.onopen = init();

function sendSignal(signal) {
    if (signalingWebsocket.readyState == 1) {
        signalingWebsocket.send(JSON.stringify(signal));
    }
};

/*
 * Initialize
 */
function init() {
    console.log("Connected to signaling endpoint. Now initializing.");    
    preparePeerConnection();
    displayLocalStreamAndSignal(true);

};

/*
 * Prepare RTCPeerConnection & setup event handlers.
 */
function preparePeerConnection() {
    
     // Using free public google STUN server.
    const configuration = {
        iceServers: [{
            urls: 'stun:stun.l.google.com:19302'
        }]
    };

    // Prepare peer connection object
    peerConnection = new RTCPeerConnection(configuration);
    peerConnection.onnegotiationneeded = async () => {
        console.log('onnegotiationneeded');
        sendOfferSignal();
    };
    peerConnection.onicecandidate = function(event) {
        if (event.candidate) {
        	sendSignal(event);
        }
    };
    
    /*
	 * Track other participant's remote stream & display in UI when available.
	 * 
	 * This is how other participant's video & audio will start showing up on my
	 * browser as soon as his local stream added to track of peer connection in
	 * his UI.
	 */
    peerConnection.addEventListener('track', displayRemoteStream);

};

/*
 * Display my local webcam & audio on UI.
 */
async function displayLocalStreamAndSignal(firstTime) {
    console.log('Requesting local stream');
    const localVideo = document.getElementById('localVideo');
    let localStream;
    try {
        // Capture local video & audio stream & set to local <video> DOM
        // element
        const stream = await navigator.mediaDevices.getUserMedia({
            audio: true,
            video: true
        });
        console.log('Received local stream');
        localVideo.srcObject = stream;
        localStream = stream;
        logVideoAudioTrackInfo(localStream);

        // For first time, add local stream to peer connection.
        if (firstTime) {
            setTimeout(
                function() {
                    addLocalStreamToPeerConnection(localStream);
                }, 2000);
        }

        // Send offer signal to signaling server endpoint.
        sendOfferSignal();

    } catch (e) {
        alert(`getUserMedia() error: ${e.name}`);
        throw e;
    }
    console.log('Start complete');
};

/*
 * Add local webcam & audio stream to peer connection so that other
 * participant's UI will be notified using 'track' event.
 * 
 * This is how my video & audio is sent to other participant's UI.
 */
async function addLocalStreamToPeerConnection(localStream) {
    console.log('Starting addLocalStreamToPeerConnection');
    localStream.getTracks().forEach(track => peerConnection.addTrack(track, localStream));
    console.log('localStream tracks added');
};

/*
 * Display remote webcam & audio in UI.
 */
function displayRemoteStream(e) {
    console.log('displayRemoteStream');
    const remoteVideo = document.getElementById('remoteVideo');
    if (remoteVideo.srcObject !== e.streams[0]) {
        remoteVideo.srcObject = e.streams[0];
        console.log('pc2 received remote stream');
    }
};

/*
 * Send offer to signaling server. This is kind of telling server that my webcam &
 * audio is ready, so notify other participant of my information so that he can
 * view & hear me using 'track' event.
 */
function sendOfferSignal() {
    peerConnection.createOffer(function(offer) {
        sendSignal(offer);
        peerConnection.setLocalDescription(offer);
    }, function(error) {
        alert("Error creating an offer");
    });
};

/*
 * Handle the offer sent by other participant & send back answer to complete the
 * handshake.
 */
function handleOffer(offer) {
    peerConnection
        .setRemoteDescription(new RTCSessionDescription(offer));

    // create and send an answer to an offer
    peerConnection.createAnswer(function(answer) {
        peerConnection.setLocalDescription(answer);
        sendSignal(answer);
    }, function(error) {
        alert("Error creating an answer");
    });

};

/*
 * Finish the handshake by receiving the answer. Now Peer-to-peer connection is
 * established between my browser & other participant's browser. Since both
 * participants are tracking each others stream, they both will be able to view &
 * hear each other.
 */
function handleAnswer(answer) {
    peerConnection.setRemoteDescription(new RTCSessionDescription(
        answer));
    console.log("connection established successfully!!");
};

/*
 * Add received ICE candidate to connection. ICE candidate has information about
 * how to connect to remote participant's browser. In local LAN connection, ICE
 * candidate might not be generated.
 */
function handleCandidate(candidate) {
	alert("handleCandidate");
    peerConnection.addIceCandidate(new RTCIceCandidate(candidate));
};

/*
 * Logs names of your webcam & microphone to console just for FYI.
 */
function logVideoAudioTrackInfo(localStream) {
    const videoTracks = localStream.getVideoTracks();
    const audioTracks = localStream.getAudioTracks();
    if (videoTracks.length > 0) {
        console.log(`Using video device: ${videoTracks[0].label}`);
    }
    if (audioTracks.length > 0) {
        console.log(`Using audio device: ${audioTracks[0].label}`);
    }
};

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

var peerConnection;

* Setup 'leaveButton' button function.

const leaveButton = document.getElementById('leaveButton');

leaveButton.addEventListener('click', leave);

function leave() {

console.log('Ending call');

peerConnection.close();

signalingWebsocket.close();

window.location.href = './index.html';

};

* Prepare websocket for signaling server endpoint.

var signalingWebsocket = new WebSocket("ws://" + window.location.host +

"/video-conf-tutorial/signal");

signalingWebsocket.onmessage = function(msg) {

console.log("Got message", msg.data);

var signal = JSON.parse(msg.data);

switch (signal.type) {

case "offer":

handleOffer(signal);

break;

case "answer":

handleAnswer(signal);

break;

// In local network, ICE candidates might not be generated.

case "candidate":

handleCandidate(signal);

break;

default:

break;

}

};

signalingWebsocket.onopen = init();

function sendSignal(signal) {

if (signalingWebsocket.readyState == 1) {

signalingWebsocket.send(JSON.stringify(signal));

}

};

* Initialize

function init() {

console.log("Connected to signaling endpoint. Now initializing.");

preparePeerConnection();

displayLocalStreamAndSignal(true);

};

* Prepare RTCPeerConnection & setup event handlers.

function preparePeerConnection() {

// Using free public google STUN server.

const configuration = {

iceServers: [{

urls: 'stun:stun.l.google.com:19302'

}]

};

// Prepare peer connection object

peerConnection = new RTCPeerConnection(configuration);

peerConnection.onnegotiationneeded = async () => {

console.log('onnegotiationneeded');

sendOfferSignal();

};

peerConnection.onicecandidate = function(event) {

if (event.candidate) {

sendSignal(event);

}

};

* Track other participant's remote stream & display in UI when available.

* This is how other participant's video & audio will start showing up on my

* browser as soon as his local stream added to track of peer connection in

* his UI.

peerConnection.addEventListener('track', displayRemoteStream);

};

* Display my local webcam & audio on UI.

async function displayLocalStreamAndSignal(firstTime) {

console.log('Requesting local stream');

const localVideo = document.getElementById('localVideo');

let localStream;

try {

// Capture local video & audio stream & set to local <video> DOM

// element

const stream = await navigator.mediaDevices.getUserMedia({

audio: true,

video: true

});

console.log('Received local stream');

localVideo.srcObject = stream;

localStream = stream;

logVideoAudioTrackInfo(localStream);

// For first time, add local stream to peer connection.

if (firstTime) {

setTimeout(

function() {

addLocalStreamToPeerConnection(localStream);

}, 2000);

}

// Send offer signal to signaling server endpoint.

sendOfferSignal();

} catch (e) {

alert(`getUserMedia() error: ${e.name}`);

throw e;

}

console.log('Start complete');

};

* Add local webcam & audio stream to peer connection so that other

* participant's UI will be notified using 'track' event.

* This is how my video & audio is sent to other participant's UI.

async function addLocalStreamToPeerConnection(localStream) {

console.log('Starting addLocalStreamToPeerConnection');

localStream.getTracks().forEach(track => peerConnection.addTrack(track, localStream));

console.log('localStream tracks added');

};

* Display remote webcam & audio in UI.

function displayRemoteStream(e) {

console.log('displayRemoteStream');

const remoteVideo = document.getElementById('remoteVideo');

if (remoteVideo.srcObject !== e.streams[0]) {

remoteVideo.srcObject = e.streams[0];

console.log('pc2 received remote stream');

}

};

* Send offer to signaling server. This is kind of telling server that my webcam &

* audio is ready, so notify other participant of my information so that he can

* view & hear me using 'track' event.

function sendOfferSignal() {

peerConnection.createOffer(function(offer) {

sendSignal(offer);

peerConnection.setLocalDescription(offer);

}, function(error) {

alert("Error creating an offer");

});

};

* Handle the offer sent by other participant & send back answer to complete the

* handshake.

function handleOffer(offer) {

peerConnection

.setRemoteDescription(new RTCSessionDescription(offer));

// create and send an answer to an offer

peerConnection.createAnswer(function(answer) {

peerConnection.setLocalDescription(answer);

sendSignal(answer);

}, function(error) {

alert("Error creating an answer");

});

};

* Finish the handshake by receiving the answer. Now Peer-to-peer connection is

* established between my browser & other participant's browser. Since both

* participants are tracking each others stream, they both will be able to view &

* hear each other.

function handleAnswer(answer) {

peerConnection.setRemoteDescription(new RTCSessionDescription(

answer));

console.log("connection established successfully!!");

};

* Add received ICE candidate to connection. ICE candidate has information about

* how to connect to remote participant's browser. In local LAN connection, ICE

* candidate might not be generated.

function handleCandidate(candidate) {

alert("handleCandidate");

peerConnection.addIceCandidate(new RTCIceCandidate(candidate));

};

* Logs names of your webcam & microphone to console just for FYI.

function logVideoAudioTrackInfo(localStream) {

const videoTracks = localStream.getVideoTracks();

const audioTracks = localStream.getAudioTracks();

if (videoTracks.length > 0) {

console.log(`Using video device: ${videoTracks[0].label}`);

}

if (audioTracks.length > 0) {

console.log(`Using audio device: ${audioTracks[0].label}`);

}

};

Video conference in action

Now that we have code ready, we can deploy above application as a war file to any server like Tomcat. Here are few things to note before you test the application.

Please check browser compatibility of navigator.mediaDevices.getUserMedia() & choose appropriate browser to test. Below demo is using chrome browser.
For the first time when you run this application, browser will prompt you to give permission to use webcam & microphone. Provide permissions to test further.
Since we are testing this using “http://” & not “https://, android chrome browser won’t let webcam & microphone to be accessed. It won’t even prompt for permissions. So you will have to whiltelist local URL in settings of unsafely-treat-insecure-origin-as-secure by opening chrome://flags . More explanation in this stackoverflow discussion.

Here is the video which shows our video conferencing application in action.

Below is the screenshot of the setting required in android to run this application in local.

GitHub Code

All above code can be found in GitHub repository of itsallbinary.

Things to try further

You can take this as base & try to play around by adding some features on top of this. Below are few ideas.

Make application multi participant video conference.
Currently it sends signal to all participants. Change it so that user can pick who to add to video conference. Signal to only those participants.
Experiment with navigator.mediaDevices & see what all you can do with it like choose specific camera on mobile etc., try mute/unmute on audio device etc.
Try running outside local network & see if you can get it to work. You might need TURN server which is not explored in this article.

References

Thanks to icons8 for free icon. https://icons8.com/icons/set/person
Thanks to https://gist.github.com/zziuni/3741933 for list of free STUN servers.
https://stackoverflow.com/questions/34197653/getusermedia-in-chrome-47-without-using-https
https://webrtc.org/

5 Replies to “Create your own video conference web application using Java & JavaScript”

Deep says:

July 11, 2020 at 8:30 am

How did you run it, am not able to start the project

Deep says:

July 11, 2020 at 5:15 pm

Hi, I am able to run it on local machine and mobile phone as well. The only problem am facing is its not showing both participant only one participant video is visible. Any pointes for that?

Thanks

1. ravik says:
  
  September 14, 2020 at 5:58 am
  
  If everything went well then it should show both user’s video. You can first try testing you application in same computer with 2 different browsers windows. Both browser windows should show your video from same webcam.
  
  If abvoe itself is not working, something might be wrong. You can use ‘Developer Tool’ in your browser & see if websocket communication is happening correctly. Also verify that there is no error in javascript. If you see any error, then will need more info on error to analyze further. Hope this helps.
  
rr says:

July 14, 2020 at 5:03 pm

conf.js?reload=true:19 WebSocket connection to ‘ws://127.0.0.1:8081/video-conf-tutorial/signal’ failed: Error during WebSocket handshake: Unexpected response code: 404

getting this error. remote video not shown

1. ravik says:
  
  September 14, 2020 at 6:02 am
  
  404 means websocket endpoint itself is not reached. But since you are seeing this error, I assume that you did see HTML successfully which means war is deployed correctly. So I think somehow websocket endpoint is not being detected by your server.
  
  Not sure which server you are using. Please check if your server is latest & updated. Also verify server supports websocket endpoints. Also verify that your server supports “annotation” based websocket endpoints.
  
  We tested this in Apache Tomcat version 9.0 & it worked perfectly. Please try this.
  
  Hope this helps.

Its All Binary – Coding Posts, Examples, Projects & More

Site for innovative & creative blog posts, Code examples, Java release history, Other projects & much more.

5 Replies to “Create your own video conference web application using Java & JavaScript”

Leave a Reply Cancel reply

Video Conferencing Application in this article

Technologies

Design

Lets Code

Video conference in action

GitHub Code

Further Reading

Things to try further

References

5 Replies to “Create your own video conference web application using Java & JavaScript”

Leave a Reply Cancel reply