DWD | Commenter-Controlled, FB Livestreamed Performance
This project is also a work-in-progress for Project Development Studio class. Initial concept can be found here.
As of now, the project does not have a title yet, but it is literally a program to turn comments on Facebook livestream into serial data for physical actuation in a performance. Full codes can be found here & here.
The program has several parts:
Facebook Login
Pulling & storing comments from a Facebook livestream using Graph API
Displaying stored comments
Parsing comments from database and turn it into serial data
Using serial data to control mechanical arm
Part 1. Facebook Login
The first step to pull comments from a user's livestream is to create an "app" on developers.facebook.com. I created an app called "livestream_show", and put in the IP address that I have on Digital Ocean.
I used a service called xip.io and appended "xip.io" to my server's IP address because (1) Facebook only allows domain names and (2) I don't have a domain name yet when I first developed the program.
The next step is to write a web app to call FB's Graph API.
Previously I used an example that I found on Graph API, but the API call happens on client-side and the access token is left exposed (code can be found here). So I rewrote the app and used a nodeJS library from criso called "fbgraph" instead, so that the call can happen on the server-side.
Once user clicks on LOGIN button, the nodeJS server will call authorization function for Graph API and prompt the user to login. This authorization function requires redirectURL, app ID, and app secret that I got from FB developer page. I also had to define the "scope" I need from the FB user. For this project, I require access to public_profile, user_videos, user_posts.
// OAuth2 Implementation - Redirect to FB login app.get('/login', function (req, res) { console.log("redirecting to: " + authUrl); res.redirect(authUrl); });
// FB redirected here after successful login app.get('/loggedin', function (req, res) { // Access Code from Facebook console.log("Access Code: " + req.query.code); // Now "hit" Facebook again with "code" to get "Access Token" // Using the graph.authorize function to do this graph.authorize({ "client_id": fb_appID , "redirect_uri": loginRedirectUrl , "client_secret": fb_secret , "code": req.query.code }, function (err, facebookRes) {
if (err) console.log(err); // Got the access token console.log("Access Token: " + facebookRes.access_token); graph.setAccessToken(facebookRes.access_token); accessToken= facebookRes.access_token;
graph.get('/me/live_videos', function(err, liveVids) { res.render('vidlist.ejs', {liveVids:liveVids}); }); }); });
(As of now, only my Facebook account is allowed to use the "app" has not been submitted for review on Facebook)
Part 2. Pulling and storing comments
Once the login and authorization is successful, the server renders the list of live videos onto an EJS template.
If there is a live video, the user can trigger the comment tracking. Otherwise, they can see an archive of live video comments, if these live videos have been tracked previously.
<h3>LIVE</h3> <% for (var i = 0; i < liveVids.data.length; i++) { %> <% if (liveVids.data[i].status == 'LIVE') { %> start tracking video: <a href="/track?liveVideoID=<%= liveVids.data[i].id %>"> <%= liveVids.data[i].id%> </a><br /> <% } %> <% } %> <h3>ARCHIVED</h3> <% for (var i = 0; i < liveVids.data.length; i++) { %> <% if (liveVids.data[i].status != 'LIVE') { %> <a href="/find?liveVideoID=<%= liveVids.data[i].id %>"> video : <%= liveVids.data[i].id%> </a><br /> <% } %> <% } %>
Clicking on a live video link will trigger the comment tracking. The comments will be pulled every half a second and stored in a mongo database in mLab. One important thing that I learned the hard way is that FB only returns 25 comments in one call by default. I did not know how to use paging in the API call yet, so I set the parameter "limit" to 500.
var id = req.query.liveVideoID; var getURL = id+'/comments?order=reverse_chronological&limit=500';
function getLiveComments(url, id){ commentInterval = setInterval(function(){ graph.setAccessToken(accessToken); graph.get(url, function(err, resp) {
for (var i=0; i<resp.data.length; i++){ var commentID = resp.data[i].id; var commentMessage = resp.data[i].message; var culpritName = resp.data[i].from.name; var culpritID = resp.data[i].from.id;
db.submissions.save({ "_id":commentID, "liveVideoID": id, "commentMessage": commentMessage, "culpritName" : culpritName, "culpritID": culpritID }, function(err, saved) { if( err || !saved ) console.log("Not saved"); //else console.log("comment saved...iteration: "+count); }); } }); count++; console.log("comment saved...iteration: "+count); },500);
}
Part 3. Displaying Stored Comments
On the website, user can also see the comment list in reverse chronological order (newest first). Since the server stores new comments every half-second, I have an AJAX call to the server that updates the comment list on the comment page. It checks if the comment ID has already been loaded previously, before loading it onto the page.
var loadedCommentID = [];
setInterval(loadComments, 1000); loadComments();
function loadComments(){ $(document).ready(function(){ $.ajax({ url: "/updatefind?liveVideoID=<%=data[0].liveVideoID%>", dataType: 'json', success: function(data) { //console.log(data); for (var i=0; i<data.length; i++){ //console.log("index: "+i); var isAlreadySaved = false; var currentCommentID = data[i]._id; //console.log(i+" | "+data[i]._id);
if(loadedCommentID){ for (var j=0; j<loadedCommentID.length; j++){ if (loadedCommentID[j] == currentCommentID){ isAlreadySaved = true; break; } } }
if(!isAlreadySaved){ loadedCommentID.push(currentCommentID); $("#comments").prepend("<p><b>"+data[i].commentMessage+"</b><br/>"+data[i].culpritName+"</p>"); } } }, error: function() { alert("tracking has stopped!"); } }); }); }
Part 4. Parsing stored comments into serial data
I wrote a nodeJS program on my machine so that I can turn the stored comments into serial data that get sent to an Arduino. I used a package called "serialport".
How the program parses the comments depends on the physical part of the performance. In this case, I used a mechanical arm that can move in 4 directions, so the program only parses comments that have the words "up", "down", "left", and "right". These strings are turned into angle values for the servo to move.
Since the server stores new comments every half-second, this program also calls into the database every half-second. One short coming in the algorithm is that it does not have paging, so every call I have to load all the comments stored, and only parse those that have not been parsed before by way of comparing IDs.
Source code can be found here.
Part 5. Using serial data to control mechanical arm
This part is fairly straightforward (also, it's outside of the scope of DWD class). I'll just post the code here:
if (Serial.available() > 0) { // if there's serial data available input_for = Serial.parseInt(); input_wipe = Serial.parseInt(); Serial.print("input_for: "); Serial.print(input_for); Serial.print(" | "); Serial.print("input_wipe: "); Serial.println(input_wipe);
} else { input_for = 0; input_wipe = 0; }
angle_for += (input_for*2); angle_for = constrain(angle_for,min_for,max_for); angle_wipe += (input_wipe*2); angle_wipe = constrain(angle_wipe,min_wipe,max_wipe);
servoMotor_for.write(angle_for); servoMotor_wipe.write(angle_wipe); delay(50);
Things to improve on, & possible future works:
Use session to store access token and make it so that multiple user can use the program.
Web socket for a more stable stream of user input & physical affect(?)
There is an inherent 10-sec delay in FB livestream, so the association between comment and physical affect may not be obvious.
Live video mixing on processing. The live mixing could be affected by user comments as well.
Related Posts
See AllDream The ultimate goal is to create a new participatory form of performance art, amplified and/or powered by the new behaviors shaped by...
