Internship opportunities at Prophesee

PROPHESEE

Founded by the world’s leading pioneers in the field of neuromorphic vision, Prophesee develops computer vision sensors and systems for application in all fields of artificial vision. The sensor technology is inspired by biological eyes, acquiring visual information in an extremely performing yet highly efficient way. Prophesee’s disruptive vision sensor technology entirely overthrows the established paradigms of frame-based vision acquisition currently used everywhere in computer vision.

This is a great opportunity to join a dynamic company and an exciting team and to lead a paradigm shift in computer vision across many industries.

EVENT-BASED TECHNOLOGY

Prophesee designs and produces a new type of cameras that are bio-inspired and thus free themselves from the concept of images. They do not gather information with a fixed framerate but instead each pixel is captured asynchronously when needed. These bio-inspired cameras, also called event-based cameras or neuromorphic cameras, therefore have an extremely sparse output and enable, with appropriate algorithms, a real time treatment of the information at an equivalent frequency of a kHz or more. But since the data coming from the sensor are quite different from the images traditionally used in standard vision, Prophesee is also advancing the algorithmic and machine learning side of this new kind of machine vision. This enables our clients to build new applications in a wide range of domains, including industrial automation, connected devices, autonomous vehicles, augmented and virtual reality, and more.

INTERNSHIP POSITION DESCRIPTION

We are looking for passionate interns who demonstrate initiative, take ownership for project work, and exhibit a strong spirit of innovation. The ideal candidate is a curious and creative individual keen on problem-solving and with prior experience in C++ / Python programming and exposure to the Computer Vision / Image Processing / Artificial Intelligence / Machine Learning domains.

She/He will work in a mixed team of scientists and engineers to design, develop & optimize solutions to research problems. Her/His main contribution will be in creating innovative bio-inspired computer vision algorithms for specific tasks across many applications such as computational imaging, 3d sensing, robotics, localization, factory automation, smart devices, aerospace & defense, automotive, etc.

The main required skills common to MOST internship positions are the following (but not exclusively):

• Excellent programming skills in C++ and/or in Python
• Engineering background in Computer Science, Mathematics or related field
• Prior experience in projects involving at least one of the following domains:
• Algorithmic design, e.g. 3d vision, machine learning, numerical optimization, etc
• Software development, e.g. implementation, architecture, optimization, testing, porting on embedded platforms, etc
• Development operations, e.g. source code versioning, continuous integration & deployment, cloud computing, system administration, etc

R&D

Capture and Render of 3D object

Neural Radiance Fields (NeRF) have revolutionized view interpolation of 3D scenes from RGB images, which brings realistic rendering results of 3D objects [1], [2]. These methods rely on precise camera pose estimation. Recent works started leveraging the high temporal resolution of event-based cameras to jointly improve camera pose estimation and the accuracy of scene representation, as well as allowing more dynamic 3D scenes reconstruction [3]. In this internship we want to explore and evaluate the use of NeRF with an event-based camera coupled with a frame-based camera. In detail, the objective is to build a 3D object representation with realistic rendering of scenes captured by a mobile phone including an event-based camera.

During the project you will work with the R&D team on the following steps:

1. Review of the state-of-the-art of Neural Rendering methods (Structure from motion, NeRF, ...) for frame-based and event-based cameras
2. Implement an architecture to train a Neural Radiance Field from real and simulated data
3. Demonstrate the approach on mobile photography application

[1] Mildenhall, B., Srinivasan, P. P., Tancik, M., Barron, J. T., Ramamoorthi, R., & Ng, R. (2021). Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM, 65(1), 99-106.

[2] Kerbl, B., Kopanas, G., Leimkühler, T., & Drettakis, G. (2023). 3d gaussian splatting for real- time radiance field rendering. ACM Transactions on Graphics (ToG), 42(4), 1-14.

[3] Ma, Q., Paudel, D. P., Chhatkuli, A., & Van Gool, L. (2023). Deformable Neural Radiance Fields using RGB and Event Cameras. arXiv preprint arXiv:2309.08416.

BENEFITS

Salary package : 80% SMIC (9,32 €/h)
Meal vouchers : 9€ / working day (Swile card)
Working hours : 35h / week
Office location : Paris Bastille (74 rue du Faubourg Saint Antoine 75012)
Other benefits : CSE