Camera Electronics experiences

BitSim develops electronics for product companies, focusing on Imaging and Edge Computing. We see a constant influx of new sensors, interfaces and key components. With the following few words we want to tell you what we think is interesting in the market, but also bring up experiences, difficulties and things to think about. And, of  course we would be happy to discuss your specific needs and solutions.

Sensors
It can be really difficult to get sensors running with all configurations needed. Sometimes we find features that are not even documented. And the sensors often have hundreds of registers where most of them have to be configured in the correct way to get an image.

  • BitSim has developed a camera with Sony’s IMX290-sensor that has very good lighting properties, i.e. can handle difficult lighting conditions. It has 10/12-bit ADC, MIPI interface, resolution up to 1080p, up to 120 fps. Flipped sideways, the resolution becomes 1109×1945 pixels. There are also a couple of HDR variants available to enable further light enhancing functionalities.
  • FLIR’s Lepton is a relatively inexpensive IR sensor that can be used separately, or in conjunction with standard CMOS sensors to extract additional information from the image through so-called “image fusion”.
  • CCS. We at BitSim hope that sensor or module suppliers will adopt MIPI’s initiative CCS – Camera Command Set: https://mipi.org/specifications/camera-command-set. The idea is to quickly get started with a sensor with its basic functionality without specific SW drivers. A typical command set can be handle things like resolution, frame rate and exposure time, but also more advanced features such as autofocus and single or multiple HDR.

Adapter card
This is often a factor making things in the development project more complex with extremely small connectors that easily break, or become loose with a bad connection.

  • We have developed a dozen different sensor adapter cards that fit development cards from Xilinx, NXP and Technexion etc. for rapid prototyping. It is a lot to think about before these small adapter cards work well, as there are usually different types of cables, connectors and sizes needed.

Interface

  • 4K Video BitSim has implemented 4K @ 60 Video, i.e. HDMI from an FPGA. In this project, we divided the camera into two physically separated parts, Front end (Camera sensor) and Back end (processing unit) with Aurora in between, i.e. Xilinx high-speed series protocol.
  • MIPI CSI-2 We have continued the development of our own camera interface IP, which now supports FPGAs with built-in D-PHY IOs (which has the advantage that no external resistance networks or Meticom circuitry are needed), e.g. a Xilinx UltraScale+ / MPSoC.  Now you can get 2.5Gb / s per lane!Processing (Platforms & Algorithms)
    One alternative for the processing of the image chain is a combined CPU and FPGA circuit, e.g. Zynq / MPSoC with the possibility to process in C / C ++ and VHDL.
  • We have worked with Python,  C / C++ and the open image library Open CV to adapt the contents of an image. With Xilinx Vision (HLS Video Library), it is also possible to use hardware accelerated OpenCV calls.
  • Another alternative is to process in a SoC circuit, i.e. with an ARM CPU, software and built-in fixed accelerators. NXP (formerly Freescale) has had great success with the i.MX6 family. The next generation, i.MX8, has been  available for a couple  of years. We have been working with the i.MX8 for a little more a year,  and we now experience that NXP’s libraries, documentation and forums are starting to become really useful.
  • We have a complete video chain, i.e. from glass-to-glass (sensor to screen), via MIPI CSI-2, V4L and Gstreamer with H.264 compression, via Ethernet to the screen.

Please contact us if you are interested or have questions!

 

 

 

Mipi A-Phy

The first step is now taken on the new MIPI A-PHY

 

We have taken our first steps on the new MIPI A-PHY specification, aiming for the automotive industry. Ethernet is becoming strong in vehicles but it isn’t suitable for every application since there may be unpredictability in the transmission time or latency when streaming some data from a sensor to the “processor”. The A-PHY support real high-speed data streaming both from the sensors to the “processor” and to the displays.

Mipi Phy

MIPI Alliance logo

The MIPI Alliance has started a collaboration with experts from the automotive industry

BitSim is a member of the MIPI Alliance (www.mipi.org), an influential standardization organization within the mobile industry which has successfully developed a number of industry standards for various well-used and established interfaces for camera sensors, displays, storage, power and audio etc.
For some time, a collaboration has been initiated by companies active in the automotive industry, with both car manufacturers and subcontractors to enhance existing or develop new interface specifications for automotive applications.

For short distances in camera applications, MIPI CSI-2 is already used in the automotive industry. BitSim has its own IP block, Bit-MIPI CSI-2, used by customers in different industry segments.

Now, this cooperation will cover solutions for longer distances, up to 15 m, for camera and radar sensors.
Since there will be up to eight cameras in modern cars in the near future, the 40 cm that MIPI CSI-2 can handle today is not enough.

BitSim’s IP, Bit-MIPI CSI-2 used in Flir’s new generataion of thermal cameras, the EXX-Series

Flir Systems is using BitSim’s FPGA-IP, Bit-MIPI CSI-2 in their newly launched advanced thermal cameras. Read more.

The MIPI Alliance, the non-profit corporation that brings the mobile industry together, has standardized the interface between the camera/image sensor and the receiving electronics (host processor or similar) for further image processing, through a high-speed serial interface, the MIPI CSI-2.

The market for image sensors is increasing by over 10% annually, with the consumer and automotive markets leading this growth.

With MIPI CSI-2, the camera providers can reach the market faster with this standardized way of interacting with the sensors: at a faster speed, at a higher resolution, or more cost effectively.

Today, MIPI CSI-2 is also used in other areas, such as test and measurement, industrial or medical markets.

Statement from our customer FLIR

BitSim has supported us in an exemplary way, and BitSim’s IP is one important part of the complete effort that made us reach the market faster in launching our new advanced thermal imaging cameras”, says Rikard hunell, Vice President Sourcing & Strategic Planning of Flir.

Summarize of the Bit-MIPI CSI-2 IP:

1-4 data lanes
AXI4-Stream Video output
D-PHY protocol decoding included
Clock-lane/data-lanes deskew
Full High Speed/Low Power mode support
Test/debug features

To learn more about BitSim and our IPs, Read More: Here