From william at 25thandClement.com Mon May 14 12:10:14 2012 From: william at 25thandClement.com (William Ahern) Date: Mon, 14 May 2012 12:10:14 -0700 Subject: [Vorbis-dev] Memory Leak in vorbis_info_clear() Message-ID: <20120514191014.GA29334@wilbur.25thandClement.com> I'm having trouble tracking down why it leaks, but below is an example program which shows--using valgrind--that vorbis_info_clear() leaks memory if called before vorbis_dsp_clear(), but not if called after vorbis_dsp_clear(). Just compile and run under valgrind, using the -l switch to the example program to trigger a leak. Tested under OS X 10.7 and Ubuntu 12.04. This may be by design, or just, "don't do that". If it is the Wiki page should probably be edited to mention this. I tried, but it didn't look like I could make the edit. My solution is to clear in reverse order of initialization, though I've rearranged the order of things in the example program to highlight the immediate issue. /* compile: cc -Wall -o vbs vbs.c -lvorbisenc -lvorbis */ /* noleak: valgrind --leak-check=yes ./vbs */ /* leak: valgrind --leak-check=yes ./vbs -l */ #include /* getopt(3) */ #include #include static struct vorbis_info info; static struct vorbis_dsp_state dsp; static struct vorbis_comment comment; static struct vorbis_block block; void init(void) { struct { ogg_packet ident, comment, setup; } pkt; vorbis_info_init(&info); vorbis_encode_init_vbr(&info, 2, 44100, 1.0); vorbis_analysis_init(&dsp, &info); vorbis_comment_init(&comment); vorbis_analysis_headerout(&dsp, &comment, &pkt.ident, &pkt.comment, &pkt.setup); vorbis_block_init(&dsp, &block); } /* init() */ void destroy(_Bool leak) { vorbis_block_clear(&block); if (leak) vorbis_info_clear(&info); vorbis_dsp_clear(&dsp); if (!leak) vorbis_info_clear(&info); vorbis_comment_clear(&comment); } /* destroy() */ int main(int argc, char *argv[]) { _Bool leak = 0; int opt; while (-1 != (opt = getopt(argc, argv, "l"))) { switch (opt) { case 'l': leak = 1; break; } } init(); destroy(leak); return 0; } /* main() */ From mgold at ncf.ca Sat May 19 09:04:58 2012 From: mgold at ncf.ca (Michael Gold) Date: Sat, 19 May 2012 12:04:58 -0400 Subject: [Vorbis-dev] [PATCH] vcut - Fix cut failure with discarded output Message-ID: <20120519160458.GA8680@terra.rilmarder.org> Hi all, Here's a patch to fix a spurious failure in vcut. Please apply. If the user wants to drop the output (i.e. the filename is given as ".") and the cutpoint is found before open_output_file is called, drop_output will not have been set, and vcut will fail with the error "Can't produce a file starting and ending between sample positions [...]". But when no output file is desired, we can safely ignore this condition. This changes the code to call the new function will_drop_output, instead of reading s->drop_output. This function checks the filename as well as s->drop_output. -- Michael -------------- next part -------------- A non-text attachment was scrubbed... Name: vcut-first-packet.diff Type: text/x-diff Size: 1652 bytes Desc: not available Url : http://lists.xiph.org/pipermail/vorbis-dev/attachments/20120519/5abe6dd3/attachment.diff -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 198 bytes Desc: Digital signature Url : http://lists.xiph.org/pipermail/vorbis-dev/attachments/20120519/5abe6dd3/attachment.pgp From rsullivan at islandofficials.com Sat May 26 20:54:56 2012 From: rsullivan at islandofficials.com (Ryan Sullivan) Date: Sat, 26 May 2012 23:54:56 -0400 Subject: [Vorbis-dev] Thread Problem. Message-ID: I have been working on a game for a while now and noticed that using your library to stream music created a graphical problem while the screen would scroll. (certain tiles would flicker to be more specific). I found that if I don't play music this problem doesn't occur. Another developer suggested that if I can lower the priority (thread priority) of the music this could fix the problem. Before hearing about this the music was not running off it's own separate thread so the only way I could do this was by creating it's own thread to stream the music, and then the rest of the code runs within the main thread. I realized that with the code being changed to use it's own thread it gets rid of the screen flickering completely. The problem is I can't seem to get it to not crash the game. After looking at this for a while I don't believe it to be a thread stack overflow that occurs. It seems the code is not thread safe and I've tried a bunch of things with mutexes or changes to the code but have not had success with this so far. The crash will either occur in ov_read or at random points within the main thread of the game. (such as doing change animation calls or cases where the game won't crash if music isn't using it's own thread) I'm going to send basically what I hope to be the only relevant code for the music and see if anyone has any suggestions on what I can do to fix this problem. -Note: I can't exactly send all the music code, but if you need to see more let me know. To start it off the main function for the music thread looks like this: void ReadStrmData() { // Switch buffer page char *buf; if(strm.bufPage == 0) { buf = strm_buf; strm.bufPage = 1; } else { buf = strm_buf + /*STRM_BUF_PAGESIZE*/4096; strm.bufPage = 0; } long total_bytes_read = 0; while(total_bytes_read < /*STRM_BUF_PAGESIZE*/4096) { int dummy; long bytes_read = ov_read( &strm.ovf, &buf[total_bytes_read], /*STRM_BUF_PAGESIZE*/4096 - total_bytes_read, &dummy); if(bytes_read < 0) { Printf("\nov_read error"); } else if(bytes_read == 0) { // End of data if(!music_queue.empty()) { ov_clear(&strm.ovf); // Get next song strm.raw_ogg = music_queue[0]; music_queue.pop_front(); strm.pos = 0; if(strm.raw_ogg != NULL) { if(ov_open_callbacks(&strm, &strm.ovf, NULL, 0, my_callbacks) < 0) { panic("ov_open_callbacks failed - 2\n"); } } else { // NULL pointer signifies silence StopMusic(); return; } } else { // Loop ov_pcm_seek(&strm.ovf, 0); } } else { total_bytes_read += bytes_read; } } } -To explain what each variable is, the first variable in that function is strm. It's a StreamInfo structure, which looks like this: struct StreamInfo { OggVorbis_File ovf; RawOggData *raw_ogg; int pos; u32 bufPage; bool is_playing; }; -At the moment strm is just a global variable. I've tried a few other things with this such as having ReadStrmData take a StreamInfo pointer. This has been recommended since the global variable is also referenced in the main thread at points when the first song would load or the same way when a new song is loaded into a level. I am also going to show you the whole process of loading a song as well. The thing is often times I've noticed the crash still occur even when a change from one song to another does not occur. The music thread consists only of ReadStrmData calls so the only time strm would be being used at all while music isn't changing is within the music thread. That was just something I attempted but I don't believe that changed anything in terms of where the crash occurs. -strm_buf looks like this: char strm_buf[STRM_BUF_SIZE * 2] ATTRIBUTE_ALIGN(32); -Again it's another global variable. -music_queue looks like this: std::deque music_queue; -Another global variable. I'm now going to show the process of loading a song, storing it in that last mentioned deque and then how it differs when music is it's own thread and not it's own thread. The load song function looks something like this: void LoadSong(std::string name_, std::deque &raw_oggs) { g_music_fade = 16; KillStream(); music_queue.clear(); raw_oggs.clear(); //then something like this //this is really a condition here which depends on //the name variable parameter //the important part is that it loads an intro to a song //and then a looped portion of a song addAndQueueSegment("song_intro", raw_oggs); addAndQueueSegment("song_theme_loop", raw_oggs); PlayStream(); } void addAndQueueSegment(const char *name, std::deque &raw_oggs) { std::string filename; filename += name; filename += ".ogg"; raw_oggs.resize(raw_oggs.size()+1); RawOggData &last = *(raw_oggs.end()-1); loadFromPakfileIntoVector(filename.c_str(), last); QueueMusicSegment(last); } void QueueMusicSegment(RawOggData &raw_ogg) { music_queue.push_back(&raw_ogg); } void KillStream() { if(strm.is_playing) { stopMusicImpl(); //this function just stops the music ov_clear(&strm.ovf); } } void PlayStream() { // Stops if during playback if(strm.is_playing) { KillStream(); } strm.raw_ogg = music_queue[0]; music_queue.pop_front(); strm.pos = 0; if(ov_open_callbacks(&strm, &strm.ovf, NULL, 0, my_callbacks) < 0) { Printf("\nov_open_callbacks failed - 1"); } strm.is_playing = TRUE; /* Set parameters */ g_timer_value = SND_TIMER_CLOCK / ov_info(&strm.ovf, -1)->rate; u32 alarmPeriod = g_timer_value * STRM_BUF_PAGESIZE / 32U; alarmPeriod /= sizeof(s16); //----- // Reads initial stream data strm.bufPage = 0; ReadStrmData(); ReadStrmData(); // Need to do it twice to fill the buffer //---- //code here that sets up an alarm, which calls a function called SoundAlarmHandler } void SoundAlarmHandler(void *arg) { //note in the case where a separate thread is not created for the music, ReadStrmData is simply called here //ReadStrmData(); //otherwise ReadStrmData is commented out as it is here and instead it sends a message for the thread code. SendMessage(.....); } This function is the function used when the music thread is created. Of course it's not used at all when there is not a separate thread for music. static void StrmThread(void * /*arg */ ) { //---- while (1) { ReceiveMessage(...); (void)ReadStrmData(); } } -Remember when a thread is not being used for the music no crashes occur at all. Whenever music is using it's own thread crashes will either occur inside ov_read called from ReadStrmData or at random points in the main thread where it wouldn't crash otherwise if there is not a separate thread. I've have read some information I found in your documentation about using ov_open_callbacks when it comes to threads. I was not sure how to adjust that so that could potentially be related to the problem. If anyone has any suggestions on what I could change and if maybe something stands out about it not being thread safe let me know. I could also show more code or perhaps go into more detail at certain points if that helps. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.xiph.org/pipermail/vorbis-dev/attachments/20120526/8c6becbf/attachment.htm From william at 25thandClement.com Sat May 26 22:43:56 2012 From: william at 25thandClement.com (William Ahern) Date: Sat, 26 May 2012 22:43:56 -0700 Subject: [Vorbis-dev] Thread Problem. In-Reply-To: References: Message-ID: <20120527054356.GA11553@wilbur.25thandClement.com> On Sat, May 26, 2012 at 11:54:56PM -0400, Ryan Sullivan wrote: > occurs. It seems the code is not thread safe and I've tried a bunch of > things with mutexes or changes to the code but have not had success with > this so far. The crash will either occur in ov_read or at random points > within the main thread of the game. (such as doing change animation calls > or cases where the game won't crash if music isn't using it's own thread) > I'm going to send basically what I hope to be the only relevant code for > the music and see if anyone has any suggestions on what I can do to fix > this problem. AFAIK, libvorbis and libogg are passively thread safe in the sense used by Unix programmers: contexts don't access shared, mutable storage. For Windows programmers, thread-safe usually means that the library actively uses mutexes internally so that the same context can be used from multiple threads out-of-the-box. This is not the case for libvorbis and libogg. You need to explicitly synchronize use of the API over each object. I don't see any locking in the code you posted. Have you used a debugger? Valgrind, Purify, dbx, etc? Valgrind, for example, can diganose mutex issues by tracking mutex use and data access among threads. FWIW, this list is pretty much dead. We may be the only ones subscribed and paying attention. From rsullivan at islandofficials.com Sat May 26 23:24:51 2012 From: rsullivan at islandofficials.com (Ryan Sullivan) Date: Sun, 27 May 2012 02:24:51 -0400 Subject: [Vorbis-dev] Thread Problem. In-Reply-To: <20120527054356.GA11553@wilbur.25thandClement.com> References: <20120527054356.GA11553@wilbur.25thandClement.com> Message-ID: Thanks for the response. I'm fairly new with using threads as the code that ran the entire game before never needed to create a separate thread. The only thing I've tried with a mutex was locking it during the beginning of the ReadStrmData call (which is basically all the separate music thread calls) and then unlocking it at the end of that call. The reason I tried this is I noticed a while ago much of the crashes would occur during the main thread, after let's say it read in somewhere around half the 4096 bytes that ReadStrmData normally reads in. In other reads it would not fully complete that function, go back to the main thread and would basically randomly crash. "This is not the case for libvorbis and libogg. You need to explicitly synchronize use of the API over each object. I don't see any locking in the code you posted." Since at the moment I have the code so the OggVorbis_File is basically a global variable there's only one object called ovf in the code. I probably should have sent the callback functions as well. That object is passed in through ov_open_callbacks. I'm not exactly sure where mutexes would need to be used. As mentioned attempting to lock the entire ReadStrmData function until it's complete didn't seem to change anything in regards to the crash. Is it possible that multiple mutexes need to be used in side some of the call back functions. Here's some of the code I left out before: const ov_callbacks my_callbacks = { OggRead, OggSeek, OggClose, OggTell }; // Mimics fread size_t OggRead(void *ptr, size_t size, size_t nmemb, void *datasource) { StreamInfo *si = (StreamInfo *)datasource; RawOggData &raw_ogg = *si->raw_ogg; size_t bytes_to_copy = std::min(nmemb*size, raw_ogg.size() - si->pos); memcpy(ptr, &raw_ogg[si->pos], bytes_to_copy); si->pos += bytes_to_copy; return bytes_to_copy; } // Mimics fseek int OggSeek(void *datasource, ogg_int64_t pos, int whence) { StreamInfo *si = (StreamInfo *)datasource; if(whence == SEEK_SET) { si->pos = int(pos); } else if(whence == SEEK_CUR) { si->pos += int(pos); } else if(whence == SEEK_END) { si->pos = int(si->raw_ogg->size() + pos); } return 0; } // Mimics fclose int OggClose(void *) { // Nothing to do return 0; } // Mimics ftell long OggTell(void *datasource) { StreamInfo *si = (StreamInfo *)datasource; return si->pos; } } -I wasn't sure how many people still use this. I'm glad at least one person does. I just happened to find it. Thanks for the help and I'll see if I can use one of those debuggers. On Sun, May 27, 2012 at 1:43 AM, William Ahern wrote: > On Sat, May 26, 2012 at 11:54:56PM -0400, Ryan Sullivan wrote: > > > occurs. It seems the code is not thread safe and I've tried a bunch of > > things with mutexes or changes to the code but have not had success with > > this so far. The crash will either occur in ov_read or at random points > > within the main thread of the game. (such as doing change animation calls > > or cases where the game won't crash if music isn't using it's own thread) > > I'm going to send basically what I hope to be the only relevant code for > > the music and see if anyone has any suggestions on what I can do to fix > > this problem. > > AFAIK, libvorbis and libogg are passively thread safe in the sense used by > Unix programmers: contexts don't access shared, mutable storage. For > Windows > programmers, thread-safe usually means that the library actively uses > mutexes internally so that the same context can be used from multiple > threads out-of-the-box. This is not the case for libvorbis and libogg. You > need to explicitly synchronize use of the API over each object. I don't see > any locking in the code you posted. > > Have you used a debugger? Valgrind, Purify, dbx, etc? Valgrind, for > example, > can diganose mutex issues by tracking mutex use and data access among > threads. > > FWIW, this list is pretty much dead. We may be the only ones subscribed and > paying attention. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.xiph.org/pipermail/vorbis-dev/attachments/20120527/5ce6f69c/attachment-0001.htm