|
|
|
| About | ||
|
|
||
| Contents of this page |
| Hybrid's MP3Search then, MediaTrak now |
MediaTrak began as a simple perl script that searched an index that was created and maintained manually. It was first known as Hybrid's MP3Search when it made its first appearance in the fall of 1996. A semi-automated crawler was created in order to ease the pain of manual indexing. In January of 1998, the MP3Search script was disallowed on the Slip.net server due to traffic and the load the script put on the system. At that time a server was bought for under $400, it's sole purpose being for the search engine. It turned out that it didn't make a good server, crashing often when stressed and just as often when not. Another server was aquired about half a year later which this site is currently running on (see specs below). Over time it developed into a fully automated search engine consisting of a crawler, search daemon, and cgi script. The crawler, written in Perl, continously crawled sites submitted by users and updated a master database while it did so. The search daemon, written in C, loaded the database into memory and performed the actual searches. It synced with the database once every several hours. The cgi, also written in Perl, sent the search request to the daemon and processed and parsed the results for return to the user. The three components combined are well within 2,000 lines of code, all written from scratch. Unfortunately there were several downfalls of the package, among which were that both the search daemon and crawler were inefficient and unstable. Both were completely replaced in April 1999, the crawler written in Java and the search engine with a mySQL database. After several months with the SQL database, it was apparent that it wasn't suitable for this task judging by its performance. Another search daemon based on the earlier one was written, also in C. While much more stable than the first, it was simple like it, lacking anything more than basic functionality. So, the daemon was rewritten in Java to take advantage of easy multi-threading, object serialization, and I/O buffering. Both perform searches tremendously quicker than the SQL database while also using less CPU power. The Java crawler is also much more efficient than the Perl crawler in that not only is it multi-threaded, but can send entire data structures of information to the daemon at a time. Everything the crawler indexes is put in the master database immediately. This means that files can be returned as results the moment they are indexed. MediaTrak put a new face on Hybrid's MP3Search in July of 1998 while also adding more functionality and support. Unfortunately, MediaTrak had to be taken down because of loss of connection and the bad server. After finding a new host and home to serve from, MediaTrak came back in January of 1999.
|
| MediaTrak Server Specs |
MediaTrak's server is a 300Mhz Intel Celeron A based system running FreeBSD 4.3-RELEASE. It is loaded with a 256mb of memory, a 4gb EIDE hard drive, and an industry standard 1.44mb floppy drive! The search engine puts little strain at all on the cpu even while handling thousands of searches a day. The web-server is release 1.3.20 of Apache, with PHP/4.0.5 compiled in. The machine is hooked up to a 192k Symmetric Digital Subscriber Line (SDSL) and is based in San Jose, California. Because of its relatively slow connection, only html's and the scripts are hosted on this server. All images are retrieved from a personal website at Slip.net. System administrator's (and webmaster's) personal note: After having used both Linux and FreeBSD, FreeBSD is the operating system of choice. Installations have been easier and the overall functionality and performance is better. FreeBSD also supports running Linux binaries and has been known to run Linux binaries better than Linux itself.
|
| MediaTrak Author |
MediaTrak, like HybridWeb (my personal web site), was created by myself. Both sites have been created from scratch; all htmls, scripts, images (except for some banners and mini-banners), etc. Each pixel, each word, and every line of code has been a [possibly painful] production of myself. My homepage, HybridWeb, is available for the overly curious. |
| |
Use of this site constitutes acceptance of the Terms and Conditions set forth on the Terms of Use page. | |
| Copyright Notice: |
|
|
| HybridWeb, MediaTrak, and the MediaTrak logo are trademarks and service marks owned by PearlMagik. The "look" and "feel" of MediaTrak and HybridWeb are also copyrighted and are property of PearlMagik. This includes MediaTrak's and PearlMagik's color combinations, layout, and all other graphical elements. All other product and company names are the trademarks or registered trademarks of their respective holders. |
|
|
|
| Copyright © 1999, 2000 PearlMagik, All Rights Reserved. |