TODO list for client ==================== ********************** URGENT ************************ Problems: * MLdonkey loses its sources without reason. * MLdonkey client generate "Exceeding Block Boundaries" errors * Walker doesnot seem to work ? Source management: * max_sources_per_file should be the number of good sources (ie sources that have chunks we want). * Popup when clicking on a file with stats on sources * Active telnet: print the current state of files when it changes * Command 'master ' to set a server as the master * Change master server when a new server has more than 5x the number of clients and files than a master server 1) For zoggy (the GUI): 1) There is a bin as icon for "Find Friend" 2) New lines in the console are not displayed because the window don't scroll immediatly 3) Show list of file names for files being downloaded 4) Add reduced icons 2) Extended search doesnot work after connect because no ping was sent. 3) vm and uploaders doesnot work in gui 4) print source propagation footprint All the last changes should be tested carefully before committing: - Test Soulseek with new rooms The GUI * Improve the gui CPU usage * Stop sharing the widgets in the chatrooms * .zog files should be manually editable (remove default properties) LimeWire and other networks: check carefully what is replied by a client for a download. In particular, check that there are no "Server busy" errors added to the download stream. Check the is_http_ok function in limewireClients.ml. Socks 5 support Savannah motds: motd.html (to be displayed in the html frame of your browser) -> generate motd.txt (to be displayed in the help panel, at the beginning, and in the telnet login) motd.conf (changes that have to be made to the default config. For example, adding servers and peers, supermode activation parameters, etc) Redirectors: send more information to the redirectors to help other users to connect Remove UDP messages to dead servers Query and parse WEB server pages such as jiggle Reduce mldonkey memory footprint Use our own network of servers For 2.00: - Fix poll mechanism for GUIs: - Messages in DriverInterface.update_ should only be sent if gui_poll is false - Consequently, Get_ messages should receive a reply immediatly if gui_poll is true. - Add special IP addresses with no upload limit and no upload slots for good friends. - Better bandwidth control: take TCP-IP packet header size into account and UDP too. - Server: server: seedIP unused and stats not sent - Number of messages in Soulseek per room - Better anticipation of TCP bandwidth - Web Server inside each server - GUI stub for multiple GUIs - Send a message for password retry - Different users per mldonkey with their own private searches - Add preview for downloaded files - Option to completely replace the server list - Cancel and Redownload doesnot work - What happens when a shared file cannot be read ? - Can a file be shared before being enough downloaded. - Batch file of ed2k:// lines - to be inserted a line of GMain.Main.set_locale () above GMain.Main.init (). ********* For 2.1: - What about having one result per file ? - Allow upload at least in Direct Connect... Maybe in gnutella too, even if it is probably a waste of bandwidth. Upload should probably be concentrated on eDonkey2000, where it is better used. However, if we don't want to be kicked from other network, it is better to allow them to download from us. Maybe we could use priorities, but how ? Each network would have a score, and the better score would be given bandwidth first ? - Incomplete files can be automatically deleted when too old (7 days) - Add a "cp" function to change the temp directory without breaking the sparse file format (ie without increasing the size of copied files) and more: - Objects should be stored in Weak hash tables everywhere, except when they are roots (servers, downloaded files, searches). - Event list: the current mechanism for notifications to GUIs is inefficient: a slow GUI prevents other GUIs from receiving fast updates and requires lots of CPU. A better system would be to have an event fifo: Each event is stored and kept in the FIFO. When a GUI connects, it points to the beginning of the fifo. Then, events are sent to it, and he can go ahead in the fifo. When an event is obsolete, it is replaced in place by the new state. The new state indicates its new position in the fifo. When events are sent to the GUI, they must be at the indicated position in the fifo. - Client side verification of query results ********* Later: * Plugins. * Correct display of availability. * Popup * Keep (server_ip, server_port, id) for indirect connections. * Manager of shared files/directories. * Add Friends in console and WEB. * Allow hostname instead of IPs in servers and friends. * Queue uploads per chunk. * Add prefered servers. * Save options in a modular way (each server in a file ?). * Recommandation for upload. * Download priorities (what does it mean ?). * Use source groups for local downloads. * Use a cache of data to help diffusing files. * Check that program exists before trying to execute. * Clean the clients_by_num table from clients which are not useful anymore. * remove locations of downloaded files. * check that 'cancelled' files cannot be shared also in incoming/ * don't add sources to files already downloaded. * efficient management of buffers * background searches: - search and automatically download files + remove unused sources (cleanly remove them at least once per hour) + commit page to select saved name + Imported files should have permission 644.[ Bug #420 ] TODO list for server ==================== * Send a ping to all clients every minute * Implement the whole protocol for TCP clients * Implement the whole protocol for UDP clients * Implement the protocol for UDP servers * Implement indexation * Implement localisation KNOWN COMMENTS (from savannah and forum): ======================================== Browser: Mozilla/4.75 [en] (X11; U; Linux 2.4.17 i686; Nav) On the help page, it still says you can't set uploads under 1K. If a server IP along with a colon seperated port number is specified in the Server IP box, and enter hit, then it gets added. This would make copy/paste from a list of servers lots easier. Saving upstats would be nice, as well as some way of resetting it, to get a better idea of which files to keep shared. The downloads panel is a little busy, and not good for small screens (some people still have to use 640x480) Perhaps move the 'save files' to an incoming tab? (which can flash when it's got files in) On the same lines, why not show the filename/details (the one above the colour bar) in a line, with name in one box, size in the next (perhaps with bytes/Mb added), and the format in another, with the colour bar as the last item in the line, taking up the whole width of the client, with the 'connections' panel only popping up over the right-hand-side of the screen when a filename is double-clicked (or when show connections is picked from the menu) The MD4 recover might move to the incoming tab, and the ed2k file to below the filename/colourbar line, and automatically change when a new file is clicked, to show the ed2k: link. Maybe the cancel/retry-connect could move to the connections panel? On the search results tab, it would be nice to indicate number of results per file, to give at least an indication of availability. Since Pierre asked for discussions on big changes in mldonkey, I think it is time to really discuss source management. We have different constraints: 1) trying too many sources takes a lot of bandwidth, and a lot of memory 2) keeping the number of sources low takes a lot of CPU, and can prevent from finding good sources. Thus, it would be interesting to find a good compromise. My current ideas: four level of sources for each file * Normal sources: these sources have been tested recently, and they have chunks interesting for us. They take a big structure in memory, and they are limited by the max_sources_per_file option. We connect to them every 10 minutes to ask for a slot. * Emerging sources: these sources have just been announced. they could have something interesting for us. We never connected to them. They are stored in a compressed form (only IP + port). * Concurrent sources: these sources have the file, but not the chunks we want. They can be tested from time to time to see if they have new chunks. One chunk is 9MB, at 5 KB/s, it is half an hour to download. Depending on the number of chunks they had the last time, we should not try them before at least 1 hour. * Old sources: these sources have once been indicated as sources for the file. We couldn't connect to them to verify that, or at least, we could'nt connect to them for a long period. We should test them anyway, maybe once every 6 hours ? They are removed after the max_sources_age delay. When new slots are available in the normal sources, they are filled from the other kind of sources, mainly by taking emerging sources, then concurrent sources and finally old sources. After 1 to 5 failed attempts to connect, they are moved to the old sources. The first kind of sources would take a lot of memory (at least 300 bytes), whereas the other sources only take around 30 bytes. If we keep all the sources that way, 10000 sources would take: 300 normal sources = 90 kB 10000 other sources = 300 kB --> 400 kB per very popular file. MLdonkey would keep asking for sources until at least good_sources_threshold of its normal sources slots are used. What do you think of such a scheme ? - MLDonkey PS: some computations. 300 normal sources = 300 connections/10 minutes = 1 connection/2 seconds for 30 files, it is 15 connections/second 1 connection = TCP Connection = 80 bytes upload, 40 bytes download edonkey hand check ~ 200 bytes upload, 200 bytes download ~ 300 B up, 300 B down 30 files --> 15x300 = 4.5 kB/s up and down If we move to old sources after 1 failed attempt = 10 minutes, we can test 300 sources/10 minutes, so 10000 old sources/5 hours 2 failed attempts = 10 hours 3 failed attempts = 15 hours 4 failed attempts = 20 hours 5 failed attempts = 25 hours ****************************************************************** How it should work: For each file, we have: ... mutable file_sources : client Intmap.t; mutable file_emerging_sources : source Fifo.t; mutable file_concurrent_sources : source Fifo.t; mutable file_old_sources : source Intmap.t; } and source = { source_addr : Ip.t * int; mutable source_client: client_kind; } and client_kind = SourceClient of client | SourceRecord of float (* last connection attempt *) When we receive a new source from anywhere: - Never seen it: add it to 'file_emerging_sources' - Already seen: * If in concurrent_sources: do nothing * If in normal_sources: do nothing * If in emerging_sources: do nothing * If in old_sources: move it to emerging_sources The options: * max_clients_per_second Constraints: * Check all clients every 10 minutes = 600 seconds max_all_sources = 600 * max_clients_per_second What we have: * a set of files with current sources. How should it work ???? At the beginning, when mldonkey starts, all the sources are emerging. Every second, we call the function "check_sources". NEW SOURCE MANAGEMENT: ----------------------