[xiph-cvs] cvs commit: positron/doc mdb.html database.html sai.html
Stan Seibert
volsung at xiph.org
Mon May 26 14:06:45 PDT 2003
volsung 03/05/26 17:06:45
Modified: doc database.html sai.html
Added: doc mdb.html
Log:
La la la, documentation, la la la.
Revision Changes Path
1.5 +41 -9 positron/doc/database.html
Index: database.html
===================================================================
RCS file: /usr/local/cvsroot/positron/doc/database.html,v
retrieving revision 1.4
retrieving revision 1.5
diff -u -r1.4 -r1.5
--- database.html 26 May 2003 14:08:44 -0000 1.4
+++ database.html 26 May 2003 21:06:45 -0000 1.5
@@ -67,7 +67,7 @@
child database.
</p>
-<h3>File Format</h3>
+<h3>File Layout</h3>
<p>
Each root database is stored in a directory whose name is the same as
@@ -101,22 +101,54 @@
audio/recordings.sai
</pre>
+
+<h3 id="dataformat">Data Packing</h3>
+
<p>
-All database files are treated as a sequence of 16-bit words. A
-"pointer" is a word offset in a file. So a pointer with value 0
-refers to bytes 0 and 1 in the file, and a pointer with value 22
-refers to bytes 44 and 45. Null-terminated strings are padded with an
-extra null byte if they do not end on a word boundary and are
-terminated with a null word (0x0000). Integers (including pointers)
-and bitfields are always stored in big-endian format.
+Because of the nature of the DSP used in the Neuros, all database
+files are treated as a sequence of 16-bit words. This creates some
+packing issues that need to be considered when storing data in
+database files. This is handled in the following way for the 5 major
+types of data:
+
+<dl>
+
+ <dt>Bit-fields</dt><dd>Bit-fields are usually 16 bits and are stored
+ in big-endian byte order.</dd>
+
+ <dt>Integers</dt><dd>Integers are 16 or 32 bits and also stored in
+ big-endian byte order.</dd>
+
+ <dt>Pointers</dt><dd>Pointers are 32 bit integers that point to
+ offsets in a file, rather than memory locations. However, unlike
+ the pointers most programmers are used to, these pointers point at
+ 16-bit words rather than bytes. So a pointer with value 0 refers to
+ bytes 0 and 1 in a file, and a pointer with value 22 refers to
+ bytes 44 and 45.</dd>
+
+ <dt>Null-terminated strings</dt><dd>These a similar to C-style strings.
+ First, the string is null-padded at the end to make it end on a word
+ boundary. Then it is terminated with a null word, 0x0000.
+ Example: "foo" (0x66,0x6f,0x6f) would be coded as 0x66,0x6f,0x6f,0x00,0x00,0x00.
+ Data of this type is refered to as sz</em> in the tables.</dd>
+
+ <dt>Display data</dt><dd>This appears to be similar in use to a
+ string, but with the ability to include some sort of binary data.
+ For this reason, display data is made by again padding the string
+ (or whatever data) out to a word boundary with nulls, but then
+ prepending a word with the data length in words, excluding the
+ length word itself. Example: "foo" would be coded as 0x00,0x02,0x66,0x6f,0x6f,0x00.
+ Data of this type is refered to as dd</em> in the tables.</dd>
+</dl>
</p>
-<a name="dbfiles"/>
+<h3 id="dbfiles">File Formats</h3>
<ul>
<li>MDB File Format</li>
<li>SAI File Format</li>
<li>PAI File Format</li>
</ul>
+
<h3>Standard Database Field Definitions</h3>
<p><p>1.3 +4 -4 positron/doc/sai.html
Index: sai.html
===================================================================
RCS file: /usr/local/cvsroot/positron/doc/sai.html,v
retrieving revision 1.2
retrieving revision 1.3
diff -u -r1.2 -r1.3
--- sai.html 26 May 2003 14:08:44 -0000 1.2
+++ sai.html 26 May 2003 21:06:45 -0000 1.3
@@ -29,11 +29,11 @@
<tr><td>4</td><td>1</td><td>Number of entries (N)</td></tr>
<tr><td>5</td><td>3</td><td>Reserved (zero by default) </td></tr>
<tr><td>8</td><td>4</td><td>0x00000000, 0x00000000 (looks like an empty record)</td></tr>
-<tr><td>12</td><td>2</td><td>MDB pointer for record 0</td></tr>
-<tr><td>14</td><td>2</td><td>PAI pointer for record 0</td></tr>
+<tr><td>12</td><td>2</td><td>MDB pointer for record 1</td></tr>
+<tr><td>14</td><td>2</td><td>PAI pointer for record 1</td></tr>
<tr><td>...</td><td>...</td><td>...</td></tr>
-<tr><td>12+4*i</td><td>2</td><td>MDB pointer for record i</td></tr>
-<tr><td>14+4*i</td><td>2</td><td>PAI pointer for record i</td></tr>
+<tr><td>12+4*(i-1)</td><td>2</td><td>MDB pointer for record i</td></tr>
+<tr><td>14+4*(i-1)</td><td>2</td><td>PAI pointer for record i</td></tr>
<tr><td>...</td><td>...</td><td>...</td></tr>
<tr><td>12+4*(N-1)</td><td>2</td><td>MDB pointer for record N</td></tr>
<tr><td>14+4*(N-1)</td><td>2</td><td>PAI pointer for record N</td></tr>
<p><p>1.1 positron/doc/mdb.html
Index: mdb.html
===================================================================
<html>
<head>
<link rel="stylesheet" type="text/css" href="style.css" />
<style type="text/css">
table.recordlayout, table.recordlayout td,
table.recordlayout th {border: 2px black solid;
border-collapse: collapse;}
table.recordlayout {margin-left: auto; margin-right: auto}
table.recordlayout td.delim{background-color: rgb(85%,85%,85%);}
table.recordlayout td.specialdelim{background-color: rgb(95%,85%,85%);}
</style>
<title>Positron Documentation: MDB File Format</title>
</head>
<body>
<h1>MDB File Format</h1>
<p>
This document explains the format of the MDB files. MDB files store
the actual contents of a database record. The MDB file begins with a
variable length header describing the overall database configuration.
The header is followed by one or more variable length records. Every
MDB file must have (generally as its first record, though it is not
clear if this location is a requirement) a null record.
This record is used by parent databases, if they exist, to specify a
null value for one of their access keys.
</p>
<p><h2>MDB Header</h2>
<p>The names given in the table below are sometimes used in
implementations, but are not normative in any way.</p>
<table class="fielddef">
<tr><th>Name</th><th>Word Offset</th><th>Size (words)</th><th>Description</th></tr>
<tr><td>LengthOfHeader<td>0</td><td>1</td><td>Length of header in words, including this word.</td></tr>
<tr><td>Attributes</td><td>1</td><td>1</td><td>Attribute bits:
<ul>
<li>Bit 0 (isRoot) = 0 if this is a child DB, 1 if this is a root DB</li>
<li>Bit 1 (isRemovableChildDB) = 0 if non-removable, 1 if removable.</li>
<p>Records in a removable DB can be deleted even if there are records in
the parent DB still referring to them. In a non-removable DB, this
is not allowed.</p>
<li>Bits 2-15 = Reserved, zero by default</li>
</td></tr>
<tr><td>Status</td><td>2</td><td>1</td><td>Status bits:
<ul>
<li>Bit 0 (isModified) = 0 if not modified since last sync, 1 if not modified.</li>
<li>Bits 1-15 = Reserved, zero by default</li>
</td></tr>
<tr><td>NumOfKeys</td><td>3</td><td>1</td><td>1 + the number of access keys, sometimes
called "rules", per record. Essentially, the primary data field is being counted here,
even though it is not an access key. (M)</td></tr>
<tr><td>NumOfFieldsPerRecord</td><td>4</td><td>1</td><td>Number of Fields per record.
Includes the primary data field, access keys, and extra info fields.</td></tr>
<tr><td>RecordStart</td><td>5</td><td>2</td><td>Word pointer to start of first record in this file</td></tr>
<tr><td>pXIM</td><td>7</td><td>2</td><td>Word pointer to the XIM data</td></tr>
<tr><td>Reserved</td><td>9</td><td>6</td><td>Reserved</td></tr>
<tr><td>DB_ID</td><td>15</td><td>1</td><td>Database ID assigned by device DB server</td></tr>
<tr><td>pDB_Name</td><td>16</td><td>2</td><td>Word pointer to DB name</td></tr>
<tr><td>pParentDBFilename</td><td>18</td><td>2</td><td>Word pointer to DB filename</td></tr>
<tr><td>pRuleName1</td><td>20</td><td>2</td><td>Word pointer to name of access key #1</td></tr>
<tr><td>pRuleFilename1</td><td>22</td><td>2</td><td>Word pointer to filename of
child MDB for access key #1</td></tr>
<tr><td>...</td><td>...</td><td>...</td><td>...</td></tr>
<tr><td>pRuleNameM</td><td>20+4*(M-1)</td><td>2</td><td>Word pointer to name of access key #M</td></tr>
<tr><td>pRuleFilenameM</td><td>22+4*(M-1)</td><td>2</td><td>Word pointer to filename of
child MDB for access key #M</td></tr>
<tr><td>...</td><td>...</td><td>...</td><td>...</td></tr>
<tr><td>DB_Name</td><td></td><td>Variable</td><td>DB name (dd format)</td></tr>
<tr><td>ParentDBFilename</td><td></td><td>Variable</td><td>DB filename (sz format)</td></tr>
<tr><td>RuleName1</td><td></td><td>Variable</td><td>Name of access key #1 (dd format)</td></tr>
<tr><td>RuleFilename1</td><td>22</td><td>Variable</td><td>Filename of child MDB file for access key #1 (sz format)</td></tr>
<tr><td>...</td><td>...</td><td>...</td><td>...</td></tr>
<tr><td>pRuleNameM</td><td></td><td>Variable</td><td>Name of access key #M (dd format)</td></tr>
<tr><td>pRuleFilenameM</td><td></td><td>Variable</td><td>Filename of
child MDB for access key #M (sz format)</td></tr>
<tr><td>...</td><td>...</td><td>...</td><td>...</td></tr>
<tr><td>XIM</td><td></td><td>Variable</td><td>XIM data</td>
<tr><td>...</td><td>...</td><td>...</td><td>...</td></tr>
<tr><td>Signature</td><td>LengthOfHeader - 2</td><td>2</td><td>Signature, "WOID"</td>
</table>
<h3 id="xim">XIM Data</h3>
<p>
XIM data is used to control the menu system on the Neuros. Various
databases are often associated with particular menus, and the XIM data
stored in the MDB header controls what menu options are available and
what they do. This documentation does not cover the structure of the
XIM data block.
</p>
<h2>MDB Records</h2>
<p>
MDB Records are composed of three types of fields (always occuring in this order):
</p>
<dl>
<dt>Primary Data</dt><dd>There is exactly one of these fields per
record. Often, in a child database, this is the only field. The
data is always stored in sz format.</dd>
<dt>Access Keys</dt><dd>There can be zero or more access keys per
database. Rather than store the actual data (like "Rock" or "FM
Radio"), a pointer to the MDB holding this data in the child
database is stored. Null values are recorded by pointed at the
<a href="#nullrec">null record</a> in the child database.</dd>
<dt>Extra Info</dt><dd>These can be zero or more extra info fields.
The format of these fields depends on the particular database. They
are usually integers or null-terminated strings.</dd>
</dl>
<p>
The number of each of these fields is fixed by the MDB header. There
is one primary data field, (NumOfKeys - 1) access keys, and
(NumOfFieldsPerRecord - NumOfKeys) extra info fields.
</p>
<p>
Fields are permitted to be multi-valued. This is used to support
fields like "Playlist" in the <a href="database.html#audio">audio
database</a>, which can have several values if a track is in multiple
playlists.
</p>
<h3>Record Layout</h3>
<p>
Since fields can have variable length, some method must be used to
mark where each field begins and ends. The method used in the MDB
format is special delimiter words:
<dl>
<dt>0x0023</dt><dd># - Field delimiter</dd>
<dt>0x0025</dt><dd>% - End of record</dd>
<dt>0x0024</dt><dd>$ - Bag delimeter. This word is used to separate
values in a field that has multiple values.</dd>
<dt>0x002F</dt><dd>/ - Escaper. If any field value contains one of
these special words (including the escaper itself), it must be
escaped using this character. For example, to store the pointer
0x00000023 in a field, you would have to write 0x0000,0x002F,0x0023.</dd>
</dl>
</p>
<h4>Normal Record</h4>
<p>
An example record in a database with all three types of fields, but
without any multivalued fields, would have the following form:
</p>
<table class="recordlayout">
<tr><td class="descriptive">Record Flag - 16 bit flag
<ul>
<li>Bit 0 (isDeleted) - 0 if an active record, 1 if a deleted record</li>
<li>Bits 1-14 - Reserved (default to 0)</li>
<li>Bit 15 - ID, always set to 1. Use this to verify you are looking a legitimate record.</li>
</ul>
</td></tr>
<tr><td>Primary Record Data (sz format)</td></tr>
<tr><td class="delim">0x0023, Field delimiter</td></tr>
<tr><td>Access Key #1 (pointer)</td></tr>
<tr><td class="delim">0x0023, Field delimiter</td></tr>
<tr><td>Access Key #2 (pointer)</td></tr>
<tr><td class="delim">0x0023, Field delimiter</td></tr>
<tr><td>Extra Info #1 (integer)</td></tr>
<tr><td class="delim">0x0023, Field delimiter</td></tr>
<tr><td>Extra Info #2 (sz data)</td></tr>
<tr><td class="delim">0x0025, End of Record</td></tr>
</table>
<h4>Record with Multivalued Field</h4>
<p>
An example record in a database with all three types of fields, but
with a multivalued fields, would have the following form:
</p>
<table class="recordlayout">
<tr><td>Record Flag</td></tr>
<tr><td>Primary Record Data (sz format)</td></tr>
<tr><td class="delim">0x0023, Field delimiter</td></tr>
<tr><td>Access Key #1, Value #1 (pointer)</td></tr>
<tr><td class="specialdelim">0x0024, Bag delimiter</td></tr>
<tr><td>Access Key #1, Value #2 (pointer)</td></tr>
<tr><td class="delim">0x0023, Field delimiter</td></tr>
<tr><td>Access Key #2 (pointer)</td></tr>
<tr><td class="delim">0x0023, Field delimiter</td></tr>
<tr><td>Extra Info #1 (integer)</td></tr>
<tr><td class="delim">0x0023, Field delimiter</td></tr>
<tr><td>Extra Info #2 (sz data)</td></tr>
<tr><td class="delim">0x0025, End of Record</td></tr>
</table>
<h4>Typical Child DB Record</h4>
<p>
Most child databases only have a primary data field, so they look like:
</p>
<table class="recordlayout">
<tr><td>Record Flag</td></tr>
<tr><td>Primary Record Data (sz format)</td></tr>
<tr><td class="delim">0x0025, End of Record</td></tr>
</table>
<h4 id="nullrec">Null Record</h4>
<p>
Every database is required to have a null record, generally the first
record. This is a degenerate record with no fields used to represent
null values in child databases. A parent will point to this record
when the associated access key has a null (or empty) value:
</p>
<table class="recordlayout">
<tr><td>0x8000, Record Flag (this record must not be deleted)</td></tr>
<tr><td class="delim">0x0025, End of Record</td></tr>
</table>
</body>
</html>
<p><p>--- >8 ----
List archives: http://www.xiph.org/archives/
Ogg project homepage: http://www.xiph.org/ogg/
To unsubscribe from this list, send a message to 'cvs-request at xiph.org'
containing only the word 'unsubscribe' in the body. No subject is needed.
Unsubscribe messages sent to the list will be ignored/filtered.
More information about the commits
mailing list