Distributed multimedia file system.

Radovici, Alexandru ; Miu, Tudor ; Dragoi, George 等

1. INTRODUCTION

Multimedia is a very important subject for virtually any computer user. Regardless of his line of business, almost any computer user will eventually make use of multimedia software in order to play music, watch movies or simply browse photos. We want to inject multimedia-servicing capabilities directly into the operating system by creating a multimedia library interface and a set of metadata operations that are integrated in the file system. Thus, requesting a service is done solely by performing a basic file input/output operation (open directory, create file, list files, etc.).Furthermore, this exposes a much easier to use programming platform since the usage of specialized metadata libraries are replaced by simple I/O operations and network functions. Such a simple API would greatly simplify the development process of multimedia-oriented applications because access to the plethora of metadata libraries is replaced with a few, very simple instructions.

Research in the field of multimedia file systems is very active. The majority of the work though focuses more on the way the multimedia content is being stored rather than its metadata handling. They provide complex storage methods using the particularities of the hardware and advanced streaming capabilities. Such examples are the Felini file system or Symphony (Shenoy et al., 1998). None of the above works focuses on the user experience neither on the programmers needs.

Our work focuses on two major issues: offering the user a better multimedia experience and making the programmer's job easier. This is done by creating a layer (wrapper) above the classic file systems.

2. GENERAL PROBLEM: FROM THE USER'S &THE PROGRAMMER'S POINT OF VIEW

Most of the nowadays multimedia software provides a media library. First of all, this means the user is able to organize his music, movies and photos and a very pleasing manner to the user. Second of all, this also usually means changing metadata like artist, song name, album title, video name, photo's information, etc., organizing files into categories, and creating playlists. Such an approach is very convenient for the end user, but, unfortunately, the media library is software dependent and, in general, is not portable across other media platforms. Once the user decides to switch to a new program or operating system, he will have to reorganize his library once again. Our idea was to create an application independent media library so that, no matter what software is used, the library would remain the same and offer the same functions. Moreover, any computer software, regardless of who created it, would benefit from the library. We decided that the best way to achieve this is to extend the operating system with a new file system. Due to the large community of programmers that have writing multimedia software, we have today so many choices when it comes to media players. From the simplest mp3 player to the most complicated integrated media systems, a user has a lot of options when deciding upon favourite program.

The main problem is that not all the software supports the same type of media files. Some recognize certain formats while others different ones. When it comes to metadata handling, the situation is worse. There are very many standards. The way that an application handles metadata depends on what metadata library its programmers have used. To address this problem and make the programmer's job easier, we have constructed a media library file system that manages the files metadata and transforms them into special files. In this way, the programmer can access and modify metadata by simply handling files.

While this approach of abstracting metadata behind files on a virtual filesystem works well for local metadata, several complications arise when trying to extend this feature to a network. There are several solutions for sharing files over a network. For instance, the user may choose from the simple file sharing solution to a network file system(Radovici & Apostol, 2009)and specialized peer-to-peer software. While these may be very convenient for a programmer, given how simple they look, these solutions are still somehow complicated for untrained user and require some technical setup effort on behalf of the user. Clearly, this does not solve the problem of how to make life easier for the end user.Instead, we are addressing this problem by distributing the multimedia file system over the network. In this way, each network computer will have mounted the same file system, thus all the users will see the same (shared) library as if it were local to the computer. Consequently, all software will have access to all the network's multimedia files as if they where stored on the local computer. With this approach, not only is the API with which the programmer has to work simplified even more to the same API for local multimedia files, but also the direct access to local and shared metadata is more intuitive for the end-user.

3. FILE SYSTEM ARCHITECTURE

3.1 System overview& the scanning engine

We have designed a file system that is in fact a thin layer above the classical file systems. Fig. 1 shows the system's architecture and functioning. Our system is built up out of there main components: the scanning engine, which scans the hard drives for new multimedia files; the file metadata database which is the core of the multimedia file system; the multimedia file system engine, which handles the system calls. Importing files into the media library is essential. This has to be done as seamlessly as possible for the user. We have built a file scanner that reads the hard drives for any known media file type. Once found, each file's metadata is read and inserted into the database.

[FIGURE 1 OMITTED]

The important new feature of the scanner is the ability to scan file once a removable media is inserted. No matter if it is a CD-ROM, DVD, or just an USB Stick, the file scanner will read its content and update the database.

Once ejected, the database will be updated so that it won't display inexistent files. If removing a media drive is easily to detect and files in the database can be deleted fast, a problem arises when a single file is deleted. It is not feasible to continuously scan all the files stored in the database and check whether they still exists. The solution used by us was a lazy check. Files are checked only when a directory is listed. If a file does not exist anymore, it will not be displayed to the user and subsequently deleted from the database. The implementation of the scanner is done using Python. This is a very flexible language, portable on almost all operating systems and very easy to develop with. The performance penalty is not very significant compared to C/C++, but the development is much easier and safer.

3.2 The file metadata database&the multimedia file system engine

The main information component is the metadata database. This is where the file system stores all the information about the multimedia content. The database is updated by the scanning engine and queried by the file system engine.

The metadata stored into the database is divided into three main categories: music, videos and photos. Each of the categories contains the unified metadata of all known formats of the file system. For music files, the file system is storing the artist name, the song name, the album title, the genre it belongs to, as well as information about the audio encoding. For videos, encoding information is stored, as well as possible the title and its director's name.ExtIf information is stored for the photos, meaning the date taken, the camera used, the lens used, etc.

The database is stored on a network server, this way each of the network computers will have access to it. This is the simplest method for all computers of having access to the file system. The implementation of the database is done using MySQL. We have chosen this system, as it is very small, resulting in an insignificant overhead in space and processing time, free and very portable (it runs on most operating systems).Moreover, MySQL has bindings for almost all programming languages, including Python (used for the file scanning engine and file system engine).If desired though, any other DBMS supporting standard SQL language may be used with very few changes to the rest of the system.

The main file system engine is placed in user space (Tanenbaum, 2004). This gives it the advantage of being able to use all user mode library functions, exactly like a normal program. In this sense, we are able to easily use database connections and sockets. The user space file system is possible due to FUSE (http://fuse.sourceforge.net).This library loads a kernel module that poses like our file system, receives the system calls from VFS (Bovet & Cesati, 2005) and redirects them to our file system engine (which in fact is a regular application).

Once received a system call, the engine queries the database for information and responds. Directory listings become actually queries into the database. Reading and writing the actual files is done by searching the actual file in the database and redirecting the system call to the normal file system where the file is stored. In this case, our system acts like a thin layer above the standard file system. The problem is somehow more complicated if the requested file is located over the network. In this case, we have implemented a simple file transfer protocol. This still needs work to be done, as it is still very basic. However, because the FUSE API is written in the C programming language (http://code.google.com/p/simplefuse), and we needed to implement a lot of high-level functions in the system, we have used the FUSE Python language binding.

4. CONCLUSIONS

We have performed tests on Ubuntu Linux (kernel 2.6.x) and Mac OS X (10.5.7).The results have shown that is indeed possible to create a virtual multimedia file system using the proposed system architecture. The files are not only properly listed under the right categories, but are also accessible for playback or visualization. Besides this, a central database provides a centralized resource for all the virtual file systems on the network, making all identical in structure. Also, a daemon constantly listens for events that are triggered when a potential new media device is connected to the computer. When this happens, it correspondingly updates the database.

We have created a multimedia file system that is centred on metadata management. In contrast with other media file systems which address mostly the problem of storage and distribution of multimedia content, our system focuses on providing the user with a unified media library and the programmer with an easier interface for reading and writing metadata. The user can now manage his multimedia library using a standard file manager instead of dedicated software. Moreover, whatever program the user will choose to work with, the media library will be the same. The programmer will be able to access multimedia metadata by simply reading and writing to files, instead of using libraries to handle metadata. Moreover, this can be done on any type of media content. This makes programming multimedia software much easier.

In the further researches, we plan to renounce the idea of a central database stored on a server. Therefore, we want to distribute the database locally to each computer. This will perhaps add complexity to synchronizing resources and overhead to network traffic, but will make the file system more tolerant to failures.

5. REFERENCES

Bovet, D.P.; Cesati, M., (2005). Understanding the Linux Kernel, 3rd Edition, O'Reilly, 0-596-000565-2,United States of America

Radovici, A.; Apostol, E. (2009). Distributed Hybrid-Storage Partially Mountable File System, Proceedings of Recent Advances in Artificial Intelligence, Knowledge Engineering and Databases, (8ed), pp. 35-40, 978-960-474-051-2, Cambridge, February 2009

Shenoy, P.J.;Goyal, P., Rao, S., Vin, H.M. (1998). Symphony: An Integrated Multimedia File System, Available from: http://www.cs.utexas.edu/users/dmcl/projects/symphony/ results1a.html, Accessed: 2008-04-20

Tanenbaum, A.S., (2004). Sisteme de operaremoderne (Modern operating systems), Byblos, 973-86699-2-8, Bucharest, Romania

*** (2009) http://fuse.sourceforge.net - File system in Userspace, Accessed on: 2009-04-10

*** (2009) http://code.google.com/p/simplefuse, Memory File System, Accessed on: 2009-04-10

RADOVICI, A[lexandru]; MIU, T[udor] & DRAGOI, G[eorge] *

* Supervisor, Mentor