I recently received the following Support Inquiry from a team setting up Transana Multiuser in Turkey.
We have a question about the video sync server.
What kind of a drive service would you recommend in the following case?
We expect the full dataset to be around 300 hours (x3 camera angles each=900h) of at least 720p videos (would require a substantial local storage space that most of us do not have in their MACs/PCs).
We want to be able to work outside of a local network of PCs (e.g., at home).
As far as I understand, Transana multiuser needs a local address for each media file we have in the dataset. However, most of the recommended video sync servers (Dropbox, Google Drive, sFTP server) do not create such an address for our dataset. As far as the IT informs us, even if the uni provides us with SSH storage, it will not sync automatically, and we will need manual downloads from that storage. How should we proceed with the media database syncing?
What do I recommend for the Video Sync Server? Unfortunately, the answer is that it depends on several factors that may be different in Turkey than they are where I live. Different universities and organizations have different rules. The “best” answer is different in different places and for different projects.
Let’s start with the crucial issue of data security and confidentiality. A lot of universities have security and confidentiality concerns with file sharing services. To be clear, I have not investigated licensing agreements or confidentiality policies in any depth, but you certainly should look into it as part of choosing a file syncing service. Laws vary from country to country, and license agreements vary from company to company. It is important that a research project get approval from both their IRB and their IT security team before using such a service with confidential data.
Services like GoogleDrive, OneDrive, iCloud, DropBox, and Box have some mechanism for accessing files as if they were on a local drive. You are correct that this functionality is required for use with Transana. Some services set up a local drive automatically. (GoogleDrive on Windows will configure as drive G: if it’s available.) Others require you to download and install a small program to automatically synchronize files in the cloud with your local hard drive such as “GoogleDrive for desktop”, “OneDrive for macOS”, “iCloud for Windows”, or “Box Sync”. It’s different for different services, but is usually pretty easy to find on their web sites. (You are correct that sFTP is a strictly manual system that requires researchers to explicitly upload and download files to move them between a local computer and cloud storage. sFTP was the way to go 15 years ago, but there are options now that automate the synchronization process. That’s why sFTP has dropped off the list of systems I talk about frequently.)
Assuming they are approved, one of the main advantages of using a file sharing service is automatic data updating. Anyone on the team can copy a file into the shared system and it will be automatically shared with the rest of the team. That can make a big difference for a team actively collecting new data or doing analyses that involve capturing screen shots from video (which creates an image file that needs to be shared). Once you get each computer configured correctly to look for files from the file sharing system’s designated location, and get your staff trained to add data to the file sharing system location as part of adding it in Transana, the process can go seamlessly.
A second *potential* advantage of file sharing systems is that they can require less overall disk space when a research team is geographically dispersed. But this advantage requires some scrutiny.
I know that Google Drive and Microsoft OneDrive have “streaming” or “on demand” options which *claim* to provide the files you need when you ask for them, thus not requiring disk space for the full media library on each computer all the time. Last time I checked, Microsoft SharePoint, DropBox and Box do not offer this feature. I’m not sure about other services. But this option relies upon very fast internet service if it’s going to work in the field. You say your data involves three simultaneous one-hour videos in a single episode. To make this work with an on-demand system, that system needs to pull about 3.6 GB of video for you to start watching an Episode or Clip for the first time. That is not what the software engineers had in mind with their “on-demand” claims. (They were thinking about word processing files and spreadsheets and maybe audio files, all of which are tiny compared to video files.) As you can imagine, such a system is very dependent on the internet speed of your local connection. I have fast 1 GB fiber-optic internet at home and have used on-demand systems successfully for single videos, but have not tried it with multiple long video files or with slower internet connections. A final issue here is about the way researchers use video. I’m analyzing a set of over 700 videos at the moment. Yesterday, I was reviewing clips for a particular theme in the data. I ended up pulling short clips from hundreds of different videos during one analysis session. If I’d been in an “on-demand” video situation, I’d have ended up with a third to a half of our data set downloaded from one day’s work. That’s just how qualitative analysis works some days. This obviously limits the disk savings possible with such a system.
If you can’t use the “streaming” or “on-demand” option, you are left copying the whole media library to each computer, which as you know can lead to drive space issues. But let’s take a deeper look at that.
Given your description of your data, let’s do some math. I have video that I’ve re-encoded to a resolution of 1024 x 576 at 2,500 kbps video bitrate using Transana’s Media Conversion tool. Why these settings? I’m looking for a balance between image size and quality on the one hand and file size on the other. You might try to go lower on the video bitrate, but it will start to become noticeable if you go too low. I’ve found that these settings give me a good video image for research purposes with a reasonable file size. With this resolution, you can watch the video full-screen on most computers without blocking or pixelation. This video comes in at just under 1.2 GB per hour.
(As a side note, I’ve worked with video files as small as 320 x 240 resolution, 360 kbps video bitrate. These files are about 200 MB per hour, and look surprisingly good within Transana when viewed as part of 3 simultaneous media file data sets. This is no longer my recommendation, however.)
So your 900 hours of video (300 hours x 3 camera), if you matched these settings, would take up a total of about 1.1 TB of space. But let’s round up to 2 TB of space to allow for bigger files, better resolution, higher video bitrate, more data, whatever. 2 TB of disk space, in the grand scheme of things, is dirt cheap these days. I don’t know about prices in Turkey, but I can get a 2 TB portable external drive for $65 on Amazon. (Yes, it’s a lot of time and work to convert 900 video files, but at least Transana’s Media Conversion tool allows you to convert multiple files at once.)
So as I see it, here are your main options:
- If the IRB and IT approve of a file sync option:
- If you can, try Google Drive’s “streaming” or OneDrive’s “on-demand” options first. For users with adequate internet speeds, this may be all you need.
- For users who experience unacceptable delays or other problems with video, switch from “streaming” to “mirroring” (in Google’s terminology) or turn off “on-demand” (in OneDrive’s language – there doesn’t seem to be a name for not-on-demand!). If these users don’t have adequate hard drive space, you may need to purchase external drives for them.
- If you end up using a system other than GoogleDrive or OneDrive, look for something like the streaming or on-demand feature, but drive mirroring or synchronization is also a reasonable option. It takes a little longer at the beginning to get all your files synchronized, but you’ll barely notice it after that.
- If you are not permitted a file sync option:
- External hard drives are a good option. They’re cheap, can be encrypted to protect confidential data against loss or theft, and they are highly portable.
- You will have to train your research staff on the correct process of bringing in new data. This includes adding new data from the field, but also creating still images through screen capture within Transana. Researchers need to make sure that all new data they add is stored within the media library, is defined correctly within Transana, and is shared with all other users through whatever mechanism the project sets up for such sharing.
- If you are not permitted an external drive option:
- Your research team is going to have to come into the office to do their work using video from the network’s shared drive space.
- Recognize that there is sometimes a really good reason for such restrictions. Some data is just too sensitive to be allowed out of the building, even when it is an inconvenience for the researchers.
I hope this clarifies things for you. Sometimes, it takes a little experimentation to determine the optimal conversion settings for media files and for file synchronization for a particular research project and a particular research team. But once you have your media files converted and the system set up correctly, Transana Multiuser works really seemlessly. If you are having troubles, please feel free to contact me for help.