[Theora-dev] FPGA implementation

[Andrey Filippov]

OK, So I've got to the point small INTRA frames work (mostly) I'll need to
verify that everything works correctly for bigger frames - there are quite
a few independent counters that synchronize operation of the compressor -
now many of them count from 0 to 0. Currently the 128x64 is the atomic
brick for my compressor - I was trying to simplify the code - anyway
sensor is 2048x1536. I'll also need to verify INTER_NOMV - (the only other
implemented), and also - how does coded/not coded logic work for the
blocks (again - some simplification due to memory limitation in FPGA and
desire to work with big sensors) - minimal unit to be coded or not is
32x32 pixels. As there is global "code all blocks" control bit, after FPGA
is programmed it is possible to minimize the program actions - on each
frame interrupt just make one register write to start 1 of 3 types of
frames - intra, inter full, and inter were coded blocks are selected by
the loaded map. Of course during this interrupt some memory pointers
should be read from FPGA - compressed data itself comes through DMA to the
circular buffer in the system memory.
 It seems I'll better do some ogg encapsulation also in the hardware so
bulk of the video data could be sent without direct CPU actions - from
FPGA to system memory by DMA, from system memory to the Ethernet MAC -
also by DMA - but that will happen with the next code upgrade, not right
now.
 And right now after I've got to some intermediate point I'll have to make
a small break. The hardware works - of course there will be some bugs
noticed and fixed similar to software later, but now the most critical
parts (at least from the hardware point of view) are tested and it seem
there is nothing left that could require complete redesign.

 I've got a stack of urgent things to do that I mostly had to put aside
for more than 4 months while working on this FPGA implementations. Among
those thing is an RFI from a big company (I can not disclose it's name
and some details) - but they need thousands of network cameras for
wideosecurity (soon and in the years to come) and they currently use (and
don't want to switch) Windows-based computers with software from
http://www.verint.com and require from the manufacturers that their
equipment be compatible with the software.

 And I've got an idea and invite you to discuss it and possibly take part
in the project - there are some funds available right away (most part
coming from the project with another not less known company who also
don't like to be identified :-) ) and more if this idea will work.

 Our first point is that it is not enough for the digital video security
systems to just reproduce the functionality of the old-style analog
systems, it should combine the following 3 features:
 1 - have high resolution so with wide angel lens it can see "everything".
No PTZ (pan/tilt/zoom), no dead zones (even covered with the dark "Domes"
so nobody can see where the camera is pointed at the moment).
 2 - have high frame rate - nobody likes to get less than has before. So
the frame rate should be the same as for the analog cameras or close to
that in the worst case.
 3 - low bit rate :-) because if you put together (1) and (2) and add
multiple cameras working in parallel - you'll get huge amount of data -
challenging for both storing and transferring over the network.

 Currently there are no cameras that combine all 3 - there are maximum 2
at once.
  And here Theora to the rescue - no motion detectors needed as the video
stream will need really low bandwidth if nothing happens. Having all the
features combined camera can really make a difference, but there is very
little software currently made, and no commercially available video
security stems can support this format. Actually, it is not just format
- it is also huge amount of data that has to be handled. And as the
camera (and it's features) is new we could not count on just
implementing some "industry standard" interface for the network cameras
- the incompatibility is "inside".

  And this is the idea for the project itself. Build a PC-based video
server, that will continuously receive up to 10-20 Ogg Theora encoded
streams from the cameras (i.e. 1280x1024 @ 30fps, 1600x1200 @ 20fps or
2048x1563 @ 12fps - limited by sensor) and write it all on disk(s).
  For the outside world this system behaves as a set of "regular" low-res
network cameras (with "digital PTZ" that they like) with the local
storage. So any standard (and established) video security software shoul
be able to talk to it and receive low-res video (produced by both
optimized decimation - up to just DC components - it stiil will be
256x192 "pixels", and windowing) Output format - just any - from motion
JPEG with coming from Netscape 4 server push technology  (used by most
such cameras with custom active X on the client) to more advanced codecs
(if CPU speed allows).
  And again - camera always sends the full resolution Ogg Theora stream
that is recorded. So this server while looking like a bunch of regular
cameras will have an important advantage - an embedded time machine. If
something will happen (traffic accident, or (popular now in many
countries) explosion - operator will always be able to go back and use
"digital PTZ" to poin the camera where he wants.

  This project seems manageable, there is no need for the perfect user
interface - server will be communicated with through the existent
software as a whole.

  Such system seems rather universal and can help to expand the use of
Theora. So even if it will not work for that particular customer it
definitely can have other uses.

  What do you think?