Subj : Re: dynamic allocation, deallocation but not heap fragmentation! To : comp.programming.threads From : jimjim Date : Sat May 14 2005 11:58 am Thank you for the reply!It was eye opening. So you mean that perhaps a a thread-per-connection design is not the optimum in my case? So you mean that I can pre-define a number of stucts and threads for use. I can utilise a linklist, which holds available threads (pthread_t *thread) for serving requests. After a thread finishes work it can return its threadid to this list for use by the main thread. When the pool of threads dries up, I can start more threads. When the linklist container's watermark falls down to, lets say, 50% I can kill threads. Can this be implemented? (however, how can I implement the threads to not die and wait for serving another request?) Or, did you mean I can have a fixed number of threads and queue up somehow client requests? How can I choose how many threads I need in this way? Thx in advance "Eric Sosman" wrote in message news:d630d7$5fq$1@news1brm.Central.Sun.COM... > > > jimjim wrote: >> Hello, >> >> I am developing a multithreaded web server. I am using pthreads. The >> pthread_create( ) takes "pthread_t *thread" & "void *arg" as two of the >> arguments. >> >> For "void *arg" I intend to pass the user-defined structure struct info >> {int clisockfd; struct sockaddr_in cliaddr }. In the main thread, after >> accept( ) returns as a client has connected, I am populating the struct >> info >> with the int clisockfd, which is the connecting client's file descriptor >> returned by accept( ), and the "struct sockaddr_in cliaddr", which as you >> guessed it is the struct passed to accept( ) and contains the client's IP >> address, port etc. >> >> My problem is that I cant really think of an effective way that I can >> create "struct info" dynamically, pass them to created threads and >> deallocate the memory when the threads terminate without perhaps >> fragmenting >> the heap after some thousands of hits. Moreover, how is it possible to >> create and delete dynamically the "pthread_t" ? >> >> Any pointers are very much wellcome. Thank you in advance. > > Are you worried about fragmentation (available memory > gets chopped into smaller and smaller pieces until they're > all too small to use), or about a memory leak (the program > keeps allocating memory but forgetting about it and failing > to recycle it)? You specifically asked about the former, but > your description sort of hints at the latter. > > If you're worried about leaking these structs, the cure > is simple: you must free() the struct when you no longer > need it. A few variations on how to do this: > > - The worker thread free()s the struct just before > terminating. This is probably simplest, so long as > there are only a very few places where the thread > can terminate. > > - The worker thread makes a local stack-resident copy > of the original, which it free()s right away. The > on-stack copy will vanish when the thread exits, so > you don't need to add free() calls to all umpty-leven > places where the thread might exit. > > On the other hand if it really *is* fragmentation you're > worried about, there isn't any 100% guaranteed way to avoid > the problem. There are some steps you can take to try to > fend it off, though: > > - Allocate these structs in blocks of N at a time instead > of individually. Don't free() a struct when you're > done with it (only 1/N of them are free()-able anyhow), > but just return them to a pool of available structs -- > a simple mutex-protected linked list will do. If the > pool is empty when the launcher thread needs a new > struct, it can allocate another N of them. (By the > way, this approach works even for N==1.) > > - Examine the allocation patterns of the rest of the > program. Are there other smallish objects that would > benefit from pooling as described above? Are there > huge objects that might be divided into smaller pieces > linked together in lists or trees or something? > > - Make every reasonable effort to free() memory as soon > as you no longer need it. The sooner you can free() > it, the sooner it has a chance to recombine with > adjacent available areas, before those areas themselves > get chopped up. > > By the way, it sounds like you're working on a thread-per- > connection design. Such designs are not usually recommended > (browse this group's archives for past discussions), and one > of the reasons has to do with all those thread stacks that > must be created and recycled. If you're worried about memory > behavior, you may want to consider the benefits of having a > small number of long-lived stacks instead of a large number > of constantly-churning stacks. I don't know how big the > thread stacks are on the system you're using, but if they're > a megabyte and you launch a thousand threads -- well, don't > you think an "extra" gig of memory might relieve some of the > stress on the rest of your program? > > -- > Eric.Sosman@sun.com > .