Jukebox Development Thread

[Guest]

1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[229]

229 Posts in 53 Topics by 24 Members
Latest Member: madsage2049

[SirLamer]

This thread shall serve as the central point of discussion regarding the development of our Jukebox music player.  I'll keep this top thread up to date with the latest top-level status of things.  I'll also post narrative updates once in awhile.  Feel free to join in and share your comments, observed bugs, etc.

Please do not post music requests here.  Use the Jukebox forum to post such requests.

To use:
Type "!jukebox" or "!jb" in chat.  The menu will pop up.  To search for something, say in chat "!jb your search query" without any special characters or other cues after the first command.  For some example content, we have a lot of TV show songs, as well as The Who and Daft Punk.

Notable features:

• Jukebox is a SourceMod plugin which adds a music streaming service to your server.  Jukebox streams MP3s from your private library to connected players.
  
• MySQL Database-driven library permits complex behind-the-scenes data logging and search abilities.
• Menu-driven interface for new users, including listings of "newest" and "most popular" tracks as well as the ability to browse content by various subject matter.
• Command line interface for familiar users, including the ability to look up content via an in-chat search query.  Results are presented in a menu structure.
• Customization settings, such as volume settings, are available to each user and are saved and restored when players return to your server.
• Includes automatic creation and playback of playlists, to permit playback of an entire subject group, such as an album.
• Playback begins without popping up the MOTD or anything similar.
• Administrator-reserved "play to all" feature, including for playlists.
• A website-based administration system.
• Permits easy upload of new tracks with automated integration into the database.  Uploaded tracks are immediately available for playback.
• Moderation of uploaded tracks permits metadata edits (track name, artist, etc.) as well as easy deletion of tracks from your webspace and the database.
• To optimize operating costs, supports automatic offloading of uploaded tracks via FTP to a "remote" file server that offers cheaper webhosting optimized for high bandwidth and low CPU needs (ie. www.canaca.com).
  
• Modular design to the plugin's script, permitting relatively easy changes that are propagated throughout the rest of the script.

Technical requirements:

• A Source engine server (any mod)
• A website server (usually offered for free by game server renters)
• A MySQL database accessible by both your web server and the game server (again, this is typical of rented servers)
• (optional) A "remote file server" for hosting of music tracks (using your main web server is supported, but most hosts won't permit such excessive bandwidth use).

Planned features:

• In-chat information on what was selected for playback.
• Integrated instructions for new users (via the MOTD page).
• Additional ability to search by genre.
• Pre-buffer of music playback software on client machine to avoid first-time hop (by loading a null playlist just after connection).
• Ability to build a playlist via track selection from several queries, then play it back (right now you can only play back a single search query's results).
• Improved means to handle the MOTD so that it displays on first connect but does not thereafter, in order to avoid playback disruptions on map change (right now I have MOTD disabled to compensate).
• System-side tracking of who is playing music to permit intelligent decisions about whether to block "play to all" commands from admins for individual clients.
• An "eavesdrop" convenience feature to let people listen to what someone else is playing without having to repeat the whole command.
• Ability for admins to adjust global settings via the menu interface.
• Automatic normalization of tracks using the Replay Gain algorithm.  This will probably require a remote computer to perform the calculations and file changes, which will be integrated via FTP.  Variable availability of such a machine will be supported, to permit the use of a home computer or another machine with casual availability.

[SirLamer]

So, I thought I would run a bit of a blog for the Jukebox development.  I will transfer this to the front end of the site (the Joomla CMS part) after I add article commenting (which I just noticed is not a standard feature!)  This will be a bit nerdy but won't be beyond anyone's range of understanding, and will discuss upcoming features and how I envision them.  Hopefully this will solicit valuable feedback from people for a quicker development schedule.

I know that the most requested feature is to for Jukebox to display track info for the songs so that people can identify what they are hearing (without harassing the admins, at least!)  This is actually harder than it seems, especially because the initial architecture for how I intended Jukebox to operate did not work.  Jukebox was originally designed for the web server to handle most of the data analysis and math and return a playlist to the game server, based on the knowledge that shared game servers have less extra computing resources available than shared web servers.  I couldn't manage to get this to work, and as a fix I transferred menu generation responsibility to the game server.  However, up until now the game server has no idea what songs the clients are listening to, since the web server still generates the final playlist.  The game server links your game client to the web server and then completely removes itself from involvement.

My focus for the past couple of weeks has been to transfer complete logic responsibility to the game server.  The game server (via a URL parameter) should just feed the web server a numerical list of song entires to play and it should blindly do so (By the way, this in itself was an enormous design challenge).  Exactly what songs and their order will be handled by the game server.  In this way, I can save the playlist and all its info into memory and keep reading from it when the song changes.

The new system works by creating what I have called "steams", which is basically a virtual sing-along of each playlist that keeps track of its predicted progress based on the track lengths.  Clients "subscribe" and unsubscibe to each stream, and the stream destroys itself either when the playlist is done or when everyone has unsubscribed from it.  Clients can even join in on a stream in progress, and it will immediately play whatever song everyone else is listening to at the time and will sync up the new user with the beginning of the next song, so that everyone can SHARE THE MAGIC or whatever.  For the most part, the user interface is unchanged, but this is a big change to how Jukebox works behind the scenes.

The big advantage of this to me is that I no longer have to design both a game server script and web server script in parallel to add to Jukebox's feature list.  It also allows me to implement features that muddle with the playlist and whatnot without worrying about the downstream code properly interpreting a bunch of new specialized commands.  In other words, more features, faster.

So you might be wondering "but, why playlists?  We always only play one song at a time".  That's true, and playlist support really does complicate the program.  In some respects, it's just a design challenge for myself.  Another reason is because I believe most albums stand a single work of art that becomes fragmented and less valuable when not listened to in its entirety as originally designed, and I don't want to deprive users of the opportunity to do that.  Also, later on Jukebox will allow users to queue up songs from several sources into a playlist or even to be played following an active playlist.  I will also be able to insert some random fun events, like fake radio commercials and stuff like that.  Future plugins will also depend on the Jukebox's playlist system for added ambience (ie. intro jingle, then context-sensitive information or sounds, then an exit jingle).  Finally, the long-promised "eavesdrop" feature will work by just attaching clients to streams, which is suddenly exceedingly easy to do.

Oh yes, you will notice that when arrogant admins such as CHEESUS and myself decide it's time to listen to Captain Planet, it will no longer interrupt you and your Rick Astley fantasies.  That is, of course, unless we want it to.

So, when you guys are repeatedly asking me and the other admins "how come it doesn't show the track names?!?!", this is why it is taking so fucking long okay?

I am going to throw up the first build of this new version for testing tonight, but I warn you, expect an assload of bugs.  I have a backup of the last stable version in case it's needed, but please do not cry in the absence of a-ha.  You will be able to take on them soon enough.

[GrimGriz]

I think this is a great project and proof that you are both dorky and awesome.

[SirLamer]

UPDATE!

The change discussed above has been implemented.

• Added display of track info (title, artist, album) in synchronization with tracks as they play in your playlist.
• Playback status (ie. who is playing what music) is now now tracked, enabling some extended operation
• When an admin uses the "play to all" command, this will not interrupt people who are already listening to music.
• When an admin uses the "force to all" command, the music will interrupt any other playback, and will disable the ability to stop playback for the duration of the playlist.  Admins may stop playback for everyone using "!jballoff".
• If "play to all" was used to stream a multi-track playlist, players who join the server in the middle of playback will join in on the music stream at the start of the next song.
• Stream memberships are tracked and restored during map changes and if a client reconnects to the server after being dropped.

Known Bugs:

• Non-admins retain the ability to stop playback when "force to all" is used.
• Corruption in the activity message that appears in the chat box.
• Proper synchronization, which is prediction-based, is impacted by the lag spike-like buffer that occurs when the first song is played.

Untested:

• Some attempted fixes for the above issues.
• A variety of operation scenarios involving multiple simultaneous music streams.

Please report bugs that you don't see mentioned here.  Thanks!

[SirLamer]

YAY!  A NEW... thing.

Jukebox now features leech protection.  A 32-bit encrypted time stamp is derived from the server's clock and included in the URL.  The keys therefore expire after a set amount of time.  This isn't NSA-level security but it just stops people from leeching songs using URLs generated from the script.  We're not having any issues, this is just future proofing in preparation for when I publish my Jukebox script.

You should notice... NOTHING.  But, if something doesn't work - notably, if music won't play - then please let me know.

Also, this change will make my script no longer work beginning early in the year 2038... in case you were wondering.

[SirLamer]

More new features!

• Added "eavesdrop" feature.  Whenever someone plays a song or playlist, the first track's title is announced to everyone.  To listen in, type "!eavesdrop" or "/eavesdrop".  In the context of a multi-track playlist, the currently playing track will play from the beginning, then playback will sync with the creator's playback at the start of the next song.  This helps people do ad-hoc karaoke or whatever.
• Further to the above, typing "!eavesdrop <player's_name>" will join the stream of the named player.  This allows people to join a friend's music stream even after someone else has started playing their own music.  Partial username matches work - ie you don't have to type out clan tags and all that.
• When changing the volume setting while a song is playing, you are offered the opportunity to restart the song.  Note that, when playing a multi-track playlist, your playback will re-sync with the original stream at the start of the next song, cutting off some of the current song.

Without anyone on the server, I haven't been able to fully test Eavesdrop very well so let me know if you discover bugs.

Regarding bugs reporting, an error occurance is often sensitive to the sequence of events.  Please report whatever sequence of steps you took that led to the error.  Thanks!

[SirLamer]

Another new topic!

Jukebox now supports sorting by Genre.  This means that the Browse by Genre menu option works.  Also, search queries (ie "!jb jazz") will return Genres now, as well.

With this, Jukebox is now feature complete.  My focus is now on completing the back-end interface that I use to manage the database as well as tieing up all remaining bugs.

The Help option will be made to point at a web page on this site which will provide help info.

[CHEESUS]

LAMER, YOU ARE A GOD.  THANK YOU FOR BUILDING SUCH AN AWESOME PROGRAM!

[SirLamer]

Thanks!  This has been an awesome learning project for me, as well, and will serve as a launching point into other features we want to develop for the server.

I should clarify that by "feature complete" I mean that I won't be adding any new features to Jukebox until I have cleaned up the existing version into a solid bug-free product.  However, feel free to post comments and requests for features, as I may decide to incorporate your ideas into Jukebox in the future.

Also, for those who expect to get upload access, I am working on a security solution which must exist before I start sharing access URLs publicly.

[SirLamer]

Another update on the latest happenings.

Anyone who plays on The Glory Hole knows that Jukebox isn't very healthy right now.  Sometimes it just refuses to work, and when more people are on the server, it's less likely to work.  This is bad because it's preventing CHEESUS from getting his funk on.

Well, the good news is that I kept a copy of Jukebox from before I made my change to the stream-based system.  It is lacking a lot of depth and has some legacy bugs in it and stuff, but it fundamentally works.  I guess that is better than... not working.  So, it's on the server now.

But, what is up with the newer Jukebox?  To be honest, I really don't understand myself.  The script bugs out at random times and even at different locations in the script.  I have even pinpointed it stopping on error-proof lines sometimes.  I just don't get it.

I think I can blame the most mysterious aspect of Jukebox, and that is the KeyValues system.  Valve designed this system that allows designers to store and manage metadata in hierarchy structures based on key-value relationships, which I am using.  Navigation through the structure is sort of blind and depends a lot on me properly predicting where in the structure the cursor is at the moment.  The script tends to bug out in vicinity to KeyValue operations, so I decided to take the bulk of them out.

Instead, the script will track everyday happening using MySQL, which is something I understand and properly more efficient anyway.

I have already written and compiled the new plugin version but I won't put it on the server until I can take some time to test it and prove it works.  If it is working properly, it will be up on the server tonight.  If not, I am keeping a copy of our last stable version on the server and I can quickly swap back to it whenever it's needed.

Thanks everyone for your help and patience!