The VINEYARD toolbox contains several frameworks for the efficient integration of heterogeneous infrastructures (e.g. FPGAs) with high-level programming frameworks like Pyhton, Java, Scala and cluster management tools like Spark. 


FPGA and GPU based accelerators have recently become first class citizens
in datacenters. Despite their high cost, however, accelerators remain
underutilized for large periods of time, as vendors prefer to dedicate them
to workloads for guaranteed QoS. At the same time, accelerator sharing is
difficult, due to vendor locked communication paths with software
applications. In this work, we modified the agents of Apache
Mesos with Vinetalk, an Apache 2.0 licensed open source accelerator middleware that abstracts the entire
communication path between OS processes and accelerator hardware without
adding significant performance overhead. The use of Vinetalk-enhanced
Mesos allows analytics pipelines, such as Apache Spark, to use for the
first time executors with heterogeneous characteristics. In addition, Vinetalk enables the ease of
integration of software applications that depend GPUs or FPGAs.

Reference Architecture:
  • Middleware
  • Apache Mesos
  • Apache Spark
  • Linux kernel
  • Nvidia CUDA
  • AMD OpenCL
  • Xilinx FPGAs