www.gandalf.user.xirium.com

Home: 1999 | 2000 | Bookmark | Sci | Tech | User

2000: 03 | 04 | 06 | 07 | 10

03: 23

various

Bergin

MacIntosh

I bought a portable MacIntosh computer. Hopefully this will make me more productive. The most obvious aspect of this will be more frequent EMail.

The Mac is a PowerBook520c and I have just discovered that it has 12MB of RAM, which is plenty, as I only intend to write text files with it (outgoing EMail, HTML, source code). Erm, if it is a PowerBook I suppose it has a PowerPC processor. I only wanted a mono, 68K Mac, but I got it as part payment for a web site update for a friend and he convinced me to get "a colour one" (and he insists on calling it a PC).

I am able to make better use of small amounts of spare time. For example, I have being writing EMail while on the bus and while on the toilet. The wonder of modern technology! [log file joke deleted.] I have not used a computer in bed since I used a Commodore VIC20.

Projects, DNS

I have a few ongoing projects they you may wish to contribute to. Hopefully, there may be CGI programming required in the future. Want to help?

There is interest in experience maintaining DNS. You may know that I have root access to two servers connected to the InterNet running DNS software. You may also know that this is the minimum requirements for administering domains. You may also know that I instructed PairNet ( http://www.pair.net/ ) to delegate subdomains of Xirium.Com to these servers. I have permission to delegate subdomains of these subdomains and friends are interested in administering their own domains.

I propose an informal DNS administration course via EMail. You would be most welcome in this group. All subdomains, servers and accounts can be provided, although you may find it more rewarding to obtain your own. Unlike most DNS administrators, you will be able to see configuration files that would usually be unavailable.

Web, CDROM, Business

I am also updating a hydroponic web site ( [domain redacted due to non payment]/ ), which is also available on a cross platform CDROM. occasional updates are also required for the sibling company that sells cannabis seeds and bongs ( http://www[organisation name redacted due to non payment].com/ ), which is also available on a cross platform CDROM. so far, more than 1300 CDROMs been distributed and only another 200 remain unused. Each CDROM is copied individually.

The reason that the CDROMs are not copied in bulk by a third party is due to flexibility. Rather than have a stock of CDROMs produced in bulk, improvements to the data can be distributed on the same day. It is not uncommon to create two or three versions of the CDROM in one day. ("Publish early, publish often.")

They are only using about 30MB on each catalogue CDROM, although this increasing due to additional images. Some of the remaining space may be used by other companies to advertise. Jeff of Esoteric believes that mail order business has significantly increased since producing CDROMs, although I am not so sure. One or more of his wholesalers have begun similar projects. Other competing and non competing retail companies are also considering. Since I have been updating the web site, accesses have tripled from Nov to Dec and doubled again from Dec to Jan. Accurate information is not available yet, although it appears that since releasing CDROMs, accesses have grown again by half and one quarter for Esoteric and Pukka respectively.

This may seems like fantastic results, although this occurred since changing web hosting provider. Given the nature of the companies, hosting in the USA was not possible. Hosting in the UK was sensible given that InterNet companies in the UK are supposedly the most advanced in Europe. It was also to reduce problems such as language barriers, payment options and legal obligations. Unfortunately, UK companies are relatively new to web hosting and seem to be using predominantly MSWindows for web hosting. This has been shown to be less reliable with twice as much downtime, according to a recent survey.

At one stage, uptime of the web sites was below 50% and was caused by a variety of problems including web server misconfiguration, which revealed that virtual hosting did not use a dedicated IP address. It is likely that most of the increase in web access is merely access returning to natural levels. In which case, there has been a significant shortfall in access. The [organisation name redacted due to non payment] web site was getting 40000 hits per month as of January and this continues to increase. (Approximately 87% of accesses were for images, which is significantly above average.) Lost sales alone exceed web hosting costs, although are lower than legal fees required to recover costs.

Jeff is quite an exception to have two companies that both profit from the web; most lose money through design and hosting. Maybe the nature of the businesses is significant. Although both web sites are promoted equally, the seed web site receives more than twice as many hits as the hydro web site. The web sites are advertised with relatively small adverts in "lads' magazines" such as Viz, FHM and Bizarre. Due to recent success and decreasing effectiveness of long term advertisement placements, Jeff reduced advertising expenditure by reducing the size of the adverts, only to find that the response rate increased. Jeff attributes some of the recent increase in mail order sales to people showing the CDROM to friends. Doubling the cost of sending information would therefore show a greater increase in the number of sales.

People share information received with their friends, although Jeff believes that it much more significant with the CDROM. More than one person, unaware of my effort, has mentioned the CDROMs to me. The fact that the CDROM is merely a copy of the web site and is therefore cross platform, increases portability and consistency while decreasing maintenance. One unexpected result of CDROM distribution is that web site accesses have actually increased. Jeff attributes this to a captive audience. Unlike viewing a web site, the user incurs no bandwidth cost when viewing the CDROM. Additionally, Jeff estimates that more of the data is viewed, partly because the data transfer rate is faster from the CDROM (see above) and partly because the user does not need to be on line.

Jeff's assessment of the situation is, as usual, quite accurate, so I will be observing his promotion policy with great interest. Currently, he appears to be diversifying, although I am not at liberty to say how.

Send-A-Quote.Com

A friend suggested starting a web site that allows quotations to be searched and EMailed. Such a web site would combine the "killer apps" of search and EMail, and large volumes of suitable data are publicly available in the form of Unix fortune cooky files. It was intended that such a web site would promote itself through the propagation of quotations. So we started Send-A-Quote.Com.

Send-A-Quote.Com ( http://www.send-a-quote.com/ ) has received about 2000000 hits so far. Listing in Yahoo ( http://www.yahoo.com/ ) generated 10000 hits per day for one week. This was due to the web site being listed as new and therefore near the top of the Yahoo Reference/Quotations web page. It also helped significantly that this section was hyperlinked directly from the Yahoo HomePage. Therefore, Send-A-Quote.Com was listed fairly prominently, two hyperlinks from one of the most accessed web pages on the web. (What volume of traffic does Yahoo get?)

The other significant traffic spike was from being prominently featured on a local television station in the USA. Apparently, the trailer for the MSNBC King5 ( http://www.king5.com/ ) television station news in the Seattle area featured Send-A-Quote.Com. (Are they toying with us?) Data transferred in one day exceeded 600MB and the following day was also elevated. I believe that bandwidth surcharges have exceeded US$200.

Two people have made money out of Send-A-Quote.Com, our Pablo, who has so far been underpaid for OCR work, and Bill Gates. Dodgy Steve, who had the idea for the web site that combined the two "killer apps" of search and EMail decided that advertising was required. (Retrospectively, the web site was probably conceived as adverting space.) We attempted to join some small advertising systems with little success. Against my advice, Dodgy Steve joined the MSN BannerSwap. I think we made a profit of about US$0.12 from that.

We write the software, design the web site, pay for InterNet access, pay for the hosting, including bandwidth surcharges, advertisers pay for the bandwidth for the advertising banners and MSN takes a 40% commission. They also advertise extensively on the BannerSwap web site, so they make a profit before you join. Additionally, it appears that MicroSoft attempted to rip the Send-A-Quote.Com data.

It is not uncommon for indexing software to access URLs of the web site. It is however standard practice to access a file stupidly named "/robots.txt" first. It is also standard practice to ignore all URLs in the "/cgi-bin/" directory. The accesses from MicroSoft did not follow this pattern. Indeed, every unique URL found was accessed until the critical script was rendered inoperable by the web hosting provider. The accesses originated from a host with substantial bandwidth on what was determined by me to be the subnet within MicroSoft that receives the company's EMail.

The web hosting provider, PairNet ( http://www.pair.net/ ), probably disabled the script before the web server became unstable and needlessly affected other customers. This activity left Send-A-Quote.Com non functional for three days and has strained relations between Send-A-Quote.Com and PairNet.

Based on comparisons with JewishNet and GeoCities, Send-A-Quote.Com was worth about US$3100000 and this may have increased. About 5% of the content is unique, user contributed content, which creates loyalty and this proportion is increasing. Less than 1:200 searches is for rude content; a typical search service has more than 1:10. The average hits per day is about 6600, of which a fairly typical proportion of one third is page views. Send-A-Quote.Com in present form has peaked.

It is for this reason that I agreed to a radical change to the database that *will* generate complaints. This change had been proposed three months ago and was the uploading of a new data that breaks existing hyperlinks. The quotes are all held a fortune cooky format, that is quotes are separated by lines with a single percent. The software generates an index file each time it is run and discovers that the index file is not present or that the time stamp on the index file is older than the fortune cooky file.

The index is held in DAP [Delta Archive Pointer] format. This consists of one file with no header and three byte offsets, low byte first, one for each quote. This allows the quote file to be up to 16MB. DAP was initially developed to compress similar versions of source code. A program collected unique lines into an archive, while DAP files pointed into the archive to specify the lines that constituted one version of source code.

Since many source code files have repeated lines, such as blank lines, or lines that only define the opening or closing of a statement block, these duplicated lines can are easily replaced with the same symbol.

The quotes are numbered (in upper case hexadecimal) and new quotations are appended to the quote file. This changes the file time stamp, which causes the file to be re-indexed when required. So, it can be seen that changing the order of the quotes affects the numbering of a given quote, which in turn affects the URL of a given quote. Unfortunately, some sites have taken to hyperlinking directly to quotes. I suspect that many people have bookmarked favourite and contributed quotes.

I told Dodgy Steve to expect complaints. (He replies to EMail and batches contributions.) I think he may underestimate the level of complaints he will receive. The disruption will increase the activity of the site, but at the expense of disgruntled users, some of which were extremely loyal. I fear that Steve will be unable to look beyond the numbers and merely see increased hits.

KickStart, AmigaFormat, iMigo

You may know that I am a member of KickStart, the Surrey Amiga User Group. At the last meeting, a representative of AmigaSoc attended and took a group photo, which will be appearing in AmigaFormat soon. Of late, there has been increased interest in creating more Amiga portables. This is achieved by converting desktop Amigas. One of the members created such an Amiga portable from an Amiga600 and a monochrome LCD.

I suggested creating a portable Amiga with a transparent plastic case. I have seen a similar project on the Web. A transparent tower case was created using plastic sheets held together with piano hinges. The tower case shown on the Web utilised an otherwise unused 12V supply to power purple fluorescent lighting, usually found as part of a high specification car hi-fi installation. Booyakasha!

The piano hinges don't move when placed along all edges of a cuboid. Additionally, the corners can be welded to create a solid subframe. Transparent plastic eliminates the need to cut a hole for the LCD screen. This simplifies construction and increases resilience of the final product.

Apart from requiring 16 feet of piano hinges, the specifications of the portable has been increasing. Stereo sound output might be a requirement. Battery level indication would be very useful. One of the less obvious suggestions requires further investigation: fuel cells.

Fuel cells store potential energy in a chemical form and it can be converted to electricity on demand. It works like a potato powered clock. Fuel cells have exceptionally good energy density, which is why they are used in space craft. Such an energy source would also be ideal for a portable computer. HydroCell UK ( http://www.fuelcells.co.uk/ ) produces custom cells made to order. The major limitation is maximum energy conversion rate. This determines how much catalyst is required. The other problem making an Amiga portable is interface to the LCD screen.

Interfacing an Amiga with a generic LCD screen is not as easy as it first appears. Most LCD screens are proprietary or accept (non standard) VGA signal input. Despite being LCD, a input signal with a 31kHz HSync is required. This requires a scan doubler between the Amiga and the screen. (This works for *any* PAL or NTSC gadget. A portable Nintendo64 has been suggested.) Sound from the phono connectors requires an amplifier circuit. The initial design will be basic but to prevent a proliferation of modules (which would look especially bad in a transparent case), I considered designing a single board with all functionality. This may require a micro-controller which may require interfacing to the computer.

With suitable software extensions (like Mac portables), this may allow proper power saving modes and automated battery monitoring. This may require interfacing via a port, although could be achieved through AutoConfig. These ideas are less important than screen and power, which I hope to implement within one month. (Y, MacOS is annoying me already.)

Ecsponent.Com

Ecsponent ( http://www.ecsponent.com/ ) is an environmentally friendly ECommerce company. The plan of the business is to promote good practice through ECommerce. As business procurement is widely expected to lead ECommerce and will remain the largest market, it is sensible to encourage good practice in this market, regardless of the number of competitors. Additionally, by encouraging and providing environmental consideration, this should further differentiate the service of Ecsponent in an active market.

For example, Ecsponent aims to provide life cycle analysis of products and provide an objective green rating. Although such information is subject to change and may not be entirely accurate, it is better than no information at all. We aim to quantify the risk of environmental damage. Some of the thinking is derived from a friend, Michael Gill, who wrote a book called Environmental Tracking. He also started the EIO [Environmental Investment Organisation]. (I think that you have met him. I know Mr Gill because I done the EIO web site, which has unfortunately been removed due to time wasters.) He derived a "blue chip" stock market fund that objectively weighted investment according to the greenness of companies.

Former attempts at environmental (and ethical) funds were subjective. Also, given the volume of shares traded by investment funds, such trading occurs within a zero sum game, minus the expenses of admin and advertising. A "blue chip" investment fund would scale, and hopefully encompass a given market. The weighting would be entirely accountable. The fund would incur minimum expense.

Had such a fund been started three years ago, it would have grown with the rise of ECommerce. If such a fund was to start now, it could provide sustainable investment, mostly immune to the problems of individual ECommerce companies. Anyone can start a share society, the requirements are minimum: have a chairperson, a treasurer and a secretary who keeps minutes of meetings. These minimal requirements prevent Ecsponent creating a professional fund with more credibility. So, the most apparent plans of Ecsponent appear fairly uninspiring. For example, implementing an ECommerce web site that quantifies the environmental soundness of the purchase alongside the financial cost.

Peter, a co-founder of Ecsponent met an employee of RedHat Software at a pub, as you do. Unbeknownst to us, [address redacted] is actually the location of RedHat's European and North African headquarters. Peter managed to book places on the first RedHat course outside of the USA. I was booked on the course before I knew about it. This was quite embarrassing because I had recently written about my dislike of "Wintel" and the negative effects of freeware Unix, which was widely distributed to KickStart and 2600/London. EMail me for a copy.

Anyhow, the Ecsponent secret skunkworks [place of covert development] shares its local pub with the RedHat staff housing. This house has been jokingly referred to as the safehouse because many of the staff regularly go abroad. I have many entertaining evenings in the pub with RedHat employees. One fellow, a Dutchman called Jules is an odd character. Through Jules, we know some details of RedHat, both locally and at headquarters. Jules joined RedHat before their IPO and may become a millionaire from his shares options if the stock market continues.

Peter has met Herman Hauser, one of the (now) millionaires of Acorn Computers. I believe that Herman was management rather than technical at Acorn. He now runs a venture capital fund, from which Peter is attempting to acquire capital.

Peter was invited by to Ester Dyson to Budapest. Ester Dyson ( http://www.icann.org/biog/dyson.htm ) is the daughter of the famous astronomer who proposed the Dyson Sphere. She is a board member of Wired Magazine and the EFF ( http://www.eff.org/ ). She has a venture capital fund called EDVenture [Ester Dyson Venture] ( http://www.edventure.com/ ). An annual conference is held every autumn and her current concerns are ECommerce (unsurprisingly) and exploiting the emerging markets of Eastern Europe. It is for this reason that the delegates and location was chosen.

Ester had heard of Ecsponent via a mutual friend. Peter corresponded with Ester by EMail and received his invitation through EMail. He initially declined, due to finances, although travel and accommodation was provided. My research indicates that successful delegates receive funding soon after this conference, but may continue to attend in order to receive additional funding. It is unlikely that we will receive funding from EDVenture, although it was encouraging to be considered by one of the most prestigious funds.

Peter and James may be subjects of a documentary for the BBC ( http://www.bbc.co.uk/ ) about obtaining venture capital in San Francisco. This has been useful because Ecsponent has obtained a client who may also be featured. I will not name the company but I will discuss some details. It seems that CompanyX are also experiencing the same challenges faced by Ecsponent. CompanyX is less technically competent than Ecsponent, although this is a minor disadvantage when most of a company's processes can be outsourced.

CompanyX was badly advised to use MicroSoft ActiveServerPages for their web site. The web site uses a small database which is not modified by the web site and therefore could have been generated from text files. Instead, the web site requires database connectivity. Since the CompanyX "database" was created using MicroSoft Access, ASP was used on the web site. This has been less than reliable and CompanyX intends to sue the former web site hosting company. The web site retains an intermittent reliability problem, for which no expert can be found.

The "database" was created within a relational database, although there were no relations between the tables. This was discovered during contracted work to optimise the underperforming database access times. Retrospectively, it would be much faster, reliable and portable to hold the data in colon delimited text files, much like the Unix password file.

I am allegedly a co-founder of Ecsponent, although this is not to be found in any (current) official documents because Ecsponent should appear open to obtain any technical expertise required. For this reason I have not received written confirmation of my stake in the business. While this may appear worrying, it is no worse than previous employment and Peter is trustworthy. The trip to San Francisco revealed that many people were uncertain why green ECommerce was needed or that it could be achieved. This may indicate that it may be unsuccessful; it may indicate that it is too advanced.

SquarePEG

At the February 2600 meeting, there was a serious suggestion which affects my proposed streaming video. It confirms my opinion on the subject. Someone should start an open source streaming video format.

There are public domain video formats, for example, Anim5. There are open source implementations of video formats, but if you observe closely, they are either support formats that are poorly defined (such as AVI and QuickTime), provide poor file size or are open source implementations of patented formats. Open source and public domain are not the same. The notable exception is MPEG which is well defined and has good file size. Unfortunately, it is *highly* asymmetric and of low quality.

Any public domain, open source video format must be free of patents and have an open source implementation. After the recent announcement of the JPEG2000 proposal, any such format must include wavelets. The fact that compression figures high been widely criticised by experts has been ignored. Many people believe the 200:1 compression claim from a reputable source.

Admittedly, the wavelet implementation provides less artifacting at the same bit rate but not to the extent claimed. One critic noted that these were the compression ratios claimed for fractal compression, which has not (yet) become widespread.

Few will remember that JPEG was largely derived from a competition. The best method, adopted by the JPEG Working Group was to split the image into blocks in YUV colour space. Tiles are subsampled from this block. Common encodings are known as 4:1:1 and 4:2:2, that is, for each four brightness tiles (Y), there are less tiles of each of the other colour components. U and V can be best described as weighted differences of Red / Green and Red / Blue. JPEG encoding is intended to minimise visible artifacting, so takes advantage of the fact that the human eye is less sensitive to colour than to brightness. Colour information is subsampled, that is colour tiles cover more area and each colour value is an average of two or more pixels.

A block consists of a number of tiles, of which brightness tiles are usually encoded at maximum quality and colour tiles are usually subsampled. In the case of 4:1:1, for every 2*2 block of Y tiles, there is one U tile and one V tile. In the case of 4:2:2, for every 2*2 block of Y tiles, there are two U tiles and two V tiles. U and V tiles are stretched in different directions to minimise (visible) artifacting. The tile size is 8*8 pixels.

Each 8*8 pixel tile is unreeled diagonally into a linear array of 64 pixels. A Discrete Cosine Transform [DCT] is applied to the array. (This is the processor intensive step.) The DCT is similar to Fourier, although only cosines are used. Half multiples of the basis frequency are used rather than sines. The amplitudes are then quantised with a simple algorithm to minimise or eliminate loss through repeatedly encoding and decoding. As expected, the amplitudes typically decay with increased frequency.

The DCT has more invariance to the position of a feature as phase information is not encoded. Additionally, the 64 amplitudes can be ordered by frequency and therefore ordered in decreasing probability of being non zero after quantisation. Unlike Fourier, values do not need to be stored in pairs. This has an overall effect of reducing the bit rate compared to Fourier.

The quantisation and decreasing probabilities allow the array of 64 amplitudes to be stored compactly. For example, there is a termination symbol and symbols to skip frequencies. In general, the array is expected to be sparse and terminates early. With the exception of the termination symbol, each symbol represents the number of frequencies to skip (4 bits) and the number of bits required to represent each amplitude (4 bits). This data is then compressed with the amplitudes. (That is vague.)

In summary, the image is split into blocks. Each block is converted into YUV colour. Tiles of 8*8 are subsampled from one colour plane in the block. Each tile is unwound into a linear array of 64 values. The DCT is performed, converting the data from samples to frequencies. The values are quantised to make a sparse array that approximates the frequencies. This data is compressed into symbols, which are compressed again.

JPEG has two compression methods, one of which is patented. The chairperson of the JPEG Working Group is named in the patent. The chairperson also published a book: an annotated version of the JPEG standard, which costs less than the official standard. Unsurprisingly, sales are good.

The compression methods are Huffman and Arithmetic (patented). Huffman compression requires a table to be calculated, making Huffman compression a two pass process. A one pass compression requires a pre-calculated table, which may be suboptimal. Arithmetic compression provides slightly better performance and is a one pass process. The improved performance is exaggerated by the differences in the implementation of the two compression methods. Additionally, optional features, such as hierarchical mode and progressive encoding, do improve quality and file size respectively, but they are rarely used.

The annotated JPEG standard complains that Adobe, part of the JPEG Working Group, breaks the standard by exceeding the arbitrary limit of 10 tiles per block. Indeed, the only thing that makes JPEG coherent is the reference implementation, used by nearly all programs. Relatively little work was done by the JPEG group and much of it was for their own benefit.

Nearly every feature of JPEG2000 is patented, although core feature are royalty free for JPEG2000 implementation only. Video is one of the patented features that will not be exempt from royalties. It will of little surprise that much of JPEG2000 was derived from one source, a company called Luratech. Apparently, Luratech specialise in wavelet compression for spy satellite applications.

The one dimensional wavelet is formed from a spiral (patented by Luratech) of the pixels and the wavelets are progressively encoded. Comments on SlashDot ( http://www.slashdot.org/ ) suggest that mixed area types will be patented and that a similar (unpatented?) system has already been in use for two years at AT&T. The example given was a newspaper, having columns of text (mono, high quality) and pictures (colour, lower quality). An encoder should identify and encode areas in suitable formats.

SquarePEG already copes very well with mixed areas. The encoder divides the image into tiles, for example, 64*64 pixels. Each tile is then speculatively and recursively subdivided down to 8*8 pixel tiles, from which the best candidate tile types are selected. When the recursion is retraced, given tiles can be matched into a simpler tile, for example, candidate tiles of similar solid colour can be averaged and replaced with one tile representing the total area. Alternatively, if the error is smaller, the candidates are instead replaced with a tile representing the area more accurately. To increase the probability matching candidate tiles, multiple tile types can be returned. Such an encoder could easily encode the newspaper example.

A tile type that I had not considered, but mentioned in the SlashDot comments is a bi-level tile type. This can be defined as being similar to the palette / true colour tile type, with the exception that it is always (1 bit) black and white, regardless of the palette. I suggest that we include such a tile type. I also suggest we include a wavelet tile in SquarePEG as a basic requirement, not optional as previously suggested. This will require more investigation, although we will be able to maintain a codec with acceptable bit rates.

Gandalf

Caution: highly flammable computer.

Re: various

hi Dean

that was an amazingly long mail! its nice of you to write a decent mail! i'll rpy l8r

im glad u seem 2 b still very much alive - i was wondering!!!

give us your tel no

[omitted]

rgds, bergin

Home: 1999 | 2000 | Bookmark | Sci | Tech | User

2000: 03 | 04 | 06 | 07 | 10

03: 23

Dean "Gandalf" Swift Mezzanine Suite 60 Urmond Road Canvey Island Essex SS8 9AF England UK Telephone: +44 1268 457360 Mobile: +44 7867 774360 EMail: dean@xirium.com