The JW Player 5 Beta is considered to be pre-release software. The documentation below can (and will) change to reflect ongoing development.
JW Player: Testing
Unit testing
Unit tests are automated and run at the function / class level. We should look at a framework like ASUnit for this. These packages can be fully covered by unit tests:
- api
- events
- parsers
- utils
Unit-testing the models and the plugins packages will prove more difficult, since they rely heavily on visual feedback.
Regression testing
Regression tests require a manual walkthrough. They are performed at the player level, in both the areas of:
- Playback of all supported file and playlist types.
- Layout, colorization and skinning.
- Playback settings and behaviours.
- Compatibility with a defined set of plugins.
- Javascript interactivity
We already have a regression test suite, both in SVN and on this site, which we should make as complete as possible.
Filetype Matrix
Here's a full matrix of filetypes we should test against the playback models we provide:
| downloads | Live stream | FLV video | MP4 video | AAC audio | MP3 audio | Images (JPG, PNG, GIF) |
| Image | X | |||||
| Sound | X | |||||
| Video | X | X | X | |||
| Youtube | X (api) | |||||
| streaming | ||||||
| HTTP (Codeshop) | X | X | ||||
| HTTP (Xmoov) | X | |||||
| RTMP (Wowza) | X | X | X | X | X | |
| RTMP (FMS) | X | X | X | X | X | |
| Smooth* (Codeshop) | X | |||||
| Smooth* (Quicktime) | X | X | ||||
| cdn's | ||||||
| Bitgravity | X | X | ||||
| EdgeCast | X | X | X | |||
| Limelight | X | |||||
| Highwinds | X | X | X | X | X | |
| Jetstream | X | X | X |
* Smooth streaming is also known as segmented streaming and chunked streaming
Items we don't have currently running are:
- An RTMP (FMS) server
- An RTMP (Wowza) server
- An Edgecast account
- A Jetstream account
We also need to regression test a number of edge cases:
- Amazon S3 signed URL (this is good because it covers a lot of edge cases: strange characters, files with no file extension, etc)
- FLV video without metadata
- RTMP disconnect after 30 idle seconds
Testing Servers
| Server / CDN | MediaProvider |
| AWS S3 | Progressive download (Video [FLV,MP4], Sound [MP3,AAC], Image [JPG,PNG]) |
| BitGravity | BOTR (HTTP [FLV,MP4]) |
| EdgeCast | FMS 3.5 (RTMP [live,FLV,MP4,MP3,AAC]) |
| Wowza | Wowza (RTMP [live,FLV,MP4,MP3,AAC]) |
| test1 | CodeShop (HTTP [FLV,MP4] / Smooth [FLV,MP4]) |
| test1 | xmoov.php (HTTP [FLV,MP4]) |
Events
- Create events and make sure data is well formed
Models
- Give models input ([valid,invalid] [[file,url],[uppercase,lowercase,camelcase]]) and make sure
- loads correctly
- dispatches events with the correct values in the correct order
- meta-data is correct
- MIME type handling
- Meta data checking
- Play back files and make sure that time events are sent and playback is smooth
- Check duration / length
RTMP
- Close on pause,
Parsers
- Give input (valid and invalid) and check config / playlist
Utils
Player
- Do some things that should dispatch events; Make sure that the correct event is received and well-formed
- Check playerLoad and starting functions to make sure they're called
- Make sure thumb comes down once playback has started and back when it's done
- Hide mouse in fullscreen
- Different colors
- [autostart,shuffle,repeat]
- Mute / volume
- Mix up plugin order and check z-index
- Set start and duration flashvars, make sure it's respected
- Add and remove listeners; Verify it's respected
Playlist
- Make sure items are ordered correctly
- Check to make sure events are thrown correctly as playlist is cycled
- Test repeat (Make sure each item is played correct number of times)
- Test adding and removing playlist items
- Playlist with streamers
- Playlist with chapters
- Playlist from youtube
MVC
Skins
- load
- check visual elements
APIs
- Add all event listeners individually, send events, check for well formed dispatches
- Add all event listeners together, send events, check for well formed dispatches
JS API
- Check playerReady to make sure its called correctly in all browsers (JS Unit)
ActionScript API
- Check sizes of known elements before and after resize / fullscreen
- Make sure buttons are added to controlbar correctly
Visual Testing
- Check a wide variety of sizes
Player
- See above
- Make sure playback is smooth
- Test redirect links
- Logo, title and no click
UI Components
- Check sizes of known elements before and after resize / fullscreen
- Check wide variety of sizes
Display
Controlbar
- Add and remove buttons
- Make sure that skins load correctly
- Check scrubber position
- Check button positions
- hide scrubber and digits if no duration
- Check resize
- Show on pause, hide on playback
- Test keyboard controls (make sure only specified keys work)
- Test mouse controls (make sure only specified buttons work)
Playlist
- Check resize
- Show on pause, hide on playback
- Make sure items are ordered correctly
Dock
- Add and remove buttons
- Check resize
Watermark
- Make sure image loads correctly
- Check size on display update
Plugins
Captions
- Load captions
- Check parsers
- test playback
LTAS + Standalones
All others
- Add all other plugins and check to see if they work / interaction
- Audiodescription and captions plugins
- Autostarter plugin
- Captions plugin (embedded captions)
- Clickproxy plugin
- HD plugin
- Livestream plugin
- Metaviewer plugin
- Revolt plugin
- Searchbar plugin
- Sharing plugin
- Snapshot plugin
