COM3001 - Matus Novak - Final Year Project

Build Status

System	Compilers	Continous Integration	Status
Windows	MinGW-w64 i686	AppVeyor
Windows	MinGW-w64 x86_64	AppVeyor
Windows	Visual Studio 2017 win32	AppVeyor
Windows	Visual Studio 2017 win64	AppVeyor
Linux	GCC 7.3.0 x64	Travis CI
OSX	Apple Clang LLVM version 8.0.0	Travis CI
Linux	GCC 5.5 x64	Circle CI
Linux	GCC 6.4 x64	Circle CI
Linux	GCC 7.3 x64	Circle CI
Linux	GCC 8.2 x64	Circle CI

Description

This is the source code of the Matus Novak's final year project for COM3001. This project is a standalone application which can encode a small http based website, encode it into an 48Khz audio, transmit over large distanced using FM transmitter, and then decode it on the other side and serve the files to the user.

Project structure

This project contains the following sub-projects (libraries) that together create the final application. The four libraries together form a client and a server applications. Where the server forms of the emitter library and the encoder library, and the client only needs the decoder and the server library.

• encoder - This is a shared library which takes a raw stream of bytes, for example a tar file, and encodes it into an audio file.
• decoder - This is a shared library which takes a raw stream of 48Khz audio samples and constructs the raw bytes back, that were created using the encoder.
• emitter - This is a shared library that consumes the raw data created by the encoder and either saves it as an mp3 file, or emitts the data to the computer sound output.
• server - This is a shared library that simply serves the decoded data to the user.

Third party libraries

This project also uses the following third party libraries:

• asio - C++11 library for creating a http server.
• zlib - C library for compression and decompression.
• Catch2 - C++14 library for unit tests only.
• argagg - For parsing command line arguments.
• dirent - Only needed on Windows as a replacement for Unix dirent.h header.

Milestones

1. Basic project structure (Done)

Create a basic project either using Conan, vcpkg, or simple git submodules for the third party libraries. Use CMake for the project configuration and C++ as the target language.

Completed! The vcpkg has been chosen as the package manager for the third party libraries, later to be replaced by simple git submodules due to lack of proper Linux support (no static libraries support). The libraries as listed in the third party libraries section above has been chosen for their simplicity and large documentation. The C++ version 2017 has been chosen as the target language for this project. The project structure and configuration works both in Linux and Windows, the OSX has not been tested at this point.

2. Encoder and Decoder phase 1 (Todo)

Create a basic encoder which takes a path to the source folder (a folder which contains the html files) and encodes them into a raw stream of audio samples. The audio samples will be targeted to 48Khz as this is the highest common sample rate used in personal computers in domestic use [1]. The next part of this milestone is to create a decoder, which takes raw data from the encoder and assembles back the raw input data. A proper unit tests must be crated during this step to create as the baseline for the next phase.

Completed! The Goertzel algorithm was used instead of the Fast Fourier Transform. The Goertzel alows to target a specific frequency, in this case two frequencies (for low and high bits). The algorithm is faster than the FFT which allows us to use this software on slower hardware. The decoding part has been programmed into blocks, similar to the GNU Radio framework.

3. Encoder and Decoder phase 2 (Todo)

Create a data link layer, in which the source input (in the encoder) is split into chunks, a data packet(s). These data packets must be small in order for them to be easily transmitted and received. The higher the packet size is, the higher the chance of the packet loss [cittation needed]. The packets will additionally contain a fixed count, the total number of packets, alongside with an index number. The decoder must be able to reject the incorrect packets and verify the integrity of the data (using a checksum).

Partially completed, there are still some things to do when splitting data into data packets.

4. Testing with FM hardware (Todo)

Testing the encoder and decoder with a real hardware. If an issue is raised, the phase 1 and 2 of the encoder and decoder must be further adjusted.

5. Noise filtering (Todo)

Implement unit testing with artificial and real noise in the audio data stream.

6. Encoder CLI (Todo)

Create a simple emitter that takes the encoded raw input as audio samples (in 48Khz) and emits them to the audio output of the machine. The encoded data must be transmited in a loop until the user decides to stop the transmission. The PortAudio library will be used for this process to access the hardware audio streams.

7. Decoder CLI + HTTP Server (Todo)

Create a simple server which takes the packets from the decoder and servers the files to the user. As an alternative, create a Windows driver (a service) which will act as an additional network device. This device will act as the internet gateway and will serve the files. Additionally, the server can cache the old files and the user will be able to view the previously transmitted version of the website. This server will also listed to the hardware audio input stream using The PortAudio. The data will be fed to the decoder library.

8. Documentation and user installation (Todo)

Write down simple documentation and create an installation (exe or msi).

9. Experiment with multiple baudrates (Todo)

Experiment with multiple data streams encoded on top of each other. For example, a higher baud rate with a lower amplitude on top of lower baudrate with higher amplitude. If the user has the perfect signal, the application will choose the higher baudrate as the source, which results in faster loading times for the website.

10. Experiment with transmitting data back (Todo)

This application only works in one way, transmitting the data to the user. An experiment can be conducted to transmit data back to the server. Essentially creating a WiFi out of FM radio transmitters and receivers.

Compiling

You will need CMake 3.1 or newer with a relevant compiler (GCC, Clang, or Visual Studio 2017). The compiler must support C++17

cd com3001
mkdir build
cd build
cmake .. -G "Visual Studio 15 2017" -DCMAKE_BUILD_TYPE=MinSizeRel -DBUILD_TESTS=ON
cmake --build . --target ALL_BUILD --config MinSizeRel

Testing

ctest --verbose -C "MinSizeRel"

References

[1] AES5-2008 (r2013): AES recommended practice for professional digital audio - Preferred sampling frequencies for applications employing pulse-code modulation (revision of AES5-2003), Audio Engineering Society, 2014-06-16