Applesauce: Difference between revisions

Latest revision as of 07:31, 4 October 2022

The Applesauce project was started in March 2017 by John K Morris of EvolutionInteractive.com. John is a well known programmer in his own right, however this is perhaps his most important contribution to the Apple II Community to date.

Applesauce will be a defining project for the Apple II Community as it will allow users to simply create exact images of their floppy media, including copy protected disks. Never before has this been possible. With some effort from the Community as a whole we can all image and save our software heritage from bitrot and being lost to history.

"I have no time to properly disseminate info to people. No time to put up a blog or anything either. 99% of the info about Applesauce exists only here on Facebook which also has a terrible search mechanism." - John K. Morris, 2017-11-18.
"Done!" - Henry S. Courbis, ReActiveMicro, 2017-11-21.
"Some time I am going to have to read this wiki." - John K. Morris, 2017-12-10.

This is the defacto Applesauce information site until John can get something official posted. Links will be added as they become available.

Release Information: On June 6th, 2018 John announced on the Apple II Enthusiasts facebook page that Applesauce is ready to be released and "The first run will be ready in 3-4 weeks and is 100 units." Prices are $199 for the standard model and $249 for the deluxe which includes 3.5" disk support. Currently John has no website or order page, and users are asked to email him at applesauce@evolutioninteractive.com in a first come first served basis of email ordering. He is planning to make more runs with the returns from the first.
Offical Website: On June 13th, 2018 John started to setup a website for Applesauce: https://applesaucefdc.com. More information will appear there as he has time and is the official home for the project (finally).
Disk II Combo Deals: ReActiveMicro is the exclusive authorized Applesauce service provider. We will be cleaning, certifying, and combo shipping Disk II drives with Applesauce orders. More details to be announced.
Be sure to subscribe to the ReActiveMicro.com Newsletter to know instantly about project release updates: https://www.reactivemicro.com/news/email-newsletter/
Also be sure to add this page to your Wiki Watchlist (upper right) to monitor updates in real time.

How Is Applesauce Useful?

In short, it's a simple solution to preserve the very complex disk structures and formats of the Apple II.

The very intuitive and simple to use UI will allow Applesauce hardware owners to create exact images of their floppy media, including copy protected disks. There will be two formats, .A2R which is "raw" data. And .WOZ which is basically the needed timing and program data from the .A2R file. This is discussed more in depth in Applesauce Image File Format (.WOZ and .A2R) the section below.

The rest of the Community will also benefit from Applesauce's new file formats and be able to run ALL Apple II software using hardware and software based emulators. Never before has this been possible. About 20% of the more popular titles have been able to be cracked, however this isn't the best way to preserve a disk and its contents. It also has its own pitfalls as discussed more in depth in Crack Versus Copy section below.

Applesauce will also allow Apple II users to start to transition to "diskless" operation. As good floppy disks start to become more scarce and originals increasingly fail we will have no choice but to look for replacement options. Images and emulators currently hold the most promise. The current generation of emulators do not support the new .WOZ format. However several authors are currently working to address this, Virtual ][ being the current one in the lead and expected to be the first to complete and release. And it's expected all emulators will support the .WOZ format in time or fall into obscurity as the Community's desire to run protected software increases due to the availability of the new disk images.

Background

In early 2017 John and Henry S. Courbis from ReActiveMicro were discussing project ideas mostly based around the Raspberry Pi. Henry sent John some hardware and after a few weeks of tinkering and discussing different projects John mentioned an idea he had involving floppy drives and backing up software. John knew Henry had worked on the EDD project and fixed several issue as a consultant for UltimateApple2 and they started to discuss how horrible and unreliable at best the EDD was in its attempt to capture the data which represents "the floppy". John started to talk about what if we attacked the problem of copying protected disks directly from the drive rather than trying to intercept the data stream from the drive under use like the EDD did. This was the start of what would become the Applesauce project.

John discovered it would not be possible to copy disks on-platform as the Apple II is too slow and too dependent on the Disk II Controller card. There would be no way to capture the timing data coming from the drive which occurs every 4 microseconds. In order to fully read a floppy disk as "raw data", also known as the flux patterns, he would need to take the drive off-platform and control it directly. The flux patterns are literally the way the magnetic particles are magnetized on the floppy medium. They are NOT bits in and of themselves. They contain timing information as well as program data.

After some research about how the Apple II encodes floppy disks John found that the only one to get things right is Jim Sather in his book Understanding the Apple II. All the other books and resources seemed to have major flaws in their interpretation of how things work and plagiarized from one another. It is quite possible that John may be one of the few people besides Wozniak and Sather to actually understand how the whole floppy subsystems actually work.

By March 2017 John had started to look into how to control the Disk II Drive and its power requirements. After some research he found the drive should be able to be controlled by a microcontroller. However most microcontrollers are not 5v tolerant and would need active and bidirectional level shifting. A proper speed microcontroller would also be needed in order to provide enough resolution to handle all the data being read from the floppy disk and allow enough time to control the drive.

A major issue would also be how to determine where the disk is in relation to the drive. This is called synchronization. IBM PC drives use a sensor to track synchronization which allows for less floppy controller processing requirements and simpler controller design to navigate tracks and sectors. The Disk II drive however does not have any synchronization mechanism which some copy protections relied on and would make copying these protected disks impossible on-platform. In order to overcome this issue John designed a sync sensor with help from Henry and added it to the arbor of the Disk II drive.

The Disk II drive's power requirements also posed some issues as the power supply would need to support universal AC input to allow for international use, and also be tri-output in order to power the Disk II drive and its logic board. John initially used a pre-built solution (as pictured) in order to reduce prototyping time. Later revisions of the project will use a custom power supply integrated onto the project's PCB.

In April John started in earnest to create the Applesauce User Interface. This will be a GUI based program for the Mac and ported to X86 and Raspberry Pi / Linux. Javier A. Rivera is assisting John with icons and other related program art work. John has also been looking for programmers to help with different parts of the project so he can better concentrate on the analysis engine which is the heart of Applesauce.

Henry's involvement has been rather limited as John has wanted to get hands on with CAD and circuit design. Although Henry has advised on several aspects of the project, John has managed to do all the labor and design work in the Proof of Concept and Alpha designs. ReActiveMicro will probably have a hand in a final design review and audit as well as overseeing assembly and distribution.

Applesauce Alpha units have been sent to several test users who have evaluated the hardware and different aspects of the software and file format.

There have been other similar projects such as Kryoflux and Catwheesel, however these projects have fallen short in several areas especially where the Apple II is concerned.

What Is Applesauce?

Applesauce comprises two parts - hardware and software.

The hardware component of the project is a "Controller Interface" based on a microcontroller that allows an Apple Disk II 5.25" disk drive to be connected via USB to a modern computer. Only Mac is currently supported, however it is expected ports of the project to X86 and Raspberry Pi (ARM) will be made available in the future. An expansion port has also been added for upgrades to the hardware - this will allow, for example, 3.5" disk imaging support in the future.

The software component runs on the modern computer and controls the Disk II Drive and captures the raw flux patterns from the inserted disk. Once this data stream is captured, the Applesauce software then processes the data for saving as disk image files.

Because Applesauce captures the data on the disk at the flux level, it is able to capture Apple II copy-protection schemes "in place". Creating an image using the flux pattern is an EXACT digital representation of the analog disk. All data on the disk is captured and recorded. In most cases the disk image could be written back to a physical disk by the Applesauce software with protection in place. Or (subject to emulation hardware or software support) it can be used with copy protection left in place on Apple II emulators or solid state Apple II disk drive emulators.

For non-protected (or previously cracked disks), Applesauce is able to capture a .dsk disk image in approximately 11s.

Applesauce will work with any Apple II disk or format that was produced: 13 sector, 16 sector, 18 sector, CP/M, ProDOS, custom OSes, etc. It does a physical capture of the fluxes on the media, so it doesn't care about the format whatsoever. If it's on the disk, Applesauce can read it. In fact, Applesauce will also image Commodore 64 and Atari (8-bit) disks just fine.

Here is an early demo of Applesauce in action from March 18, 2017: https://www.facebook.com/groups/5251478676/permalink/10156006837173677/

Flux Patterns

Magnetic media flux patterns, or just flux for short, are an inherent physical property of all magnetic media. The pattern refers to how each region of the media is magnetized. These patterns are used to store track and sector information, as well as program data unrelated to the disk operation.

All magnetic media is coated with a ferrous material. This allows small regions in the magnetic media to be used as individual magnets. The smaller the region the more "bit dense" the material is said to be. The media is then able to be used to store data. This is the basis of how all such media is created and used.

Floppy disks use a track and sector coordinate system to store information. Each track holds a set amount of sectors, and these can vary by OS and drive design. The flux patterns are created when the head of the disk drive writes to the disk. These patterns are permanent unless damaged or overwritten. Damage can occur in several ways but is most likely due to age or scratching of the media.

Individual fluxes are always in a North or South polarity as with all magnets.

Weak Bits, Bubble Bits, Fake Bits, And The MC3470

The more correct name for these kinds of data bits or phantom flux transitions should actually be "random bits" or "random transitions" and we will explain why. Technically speaking the phenomenon is called an "AGC Error" which stands for Automatic Gain Control and is actually normal behavior of the MC3470.

There has been some odd terms thrown around by different people about individual fluxes and their behavior. One such term used most commonly is "weak bit" which can mean two possible things.

Weak can refer to the state of the individual or group of flux patterns that have physically become damaged and are unable to be reliably read. This can be physically caused by damage to the media such as oxidation, mold, or the magnetic coating flaking off and becoming lodged under the drive head. As debris accumulates under the drive head scratching of the media becomes more likely. Once this kind of damage happens there is no way to repair it. Damage can also happen in the form of magnetic erasure which affects the levels of north or south saturation for a given area making it appear neutral or degaussed. If a disk is mishandled or brought too close to a magnetic source then parts of the disk can be changed randomly. This kind of issue can be repaired by rewriting the affected area and does not change location on the disk.

The other form of weak bits not caused by damage are technically called an AGC Error (Automatic Gain Control), but also called "random bits", "bubble bits", "fake bits", and "float bits" to name a few less technical terms people have used over the years. These flux patterns appear randomly when read and constantly change location, and usually in groups of two located very close to one another. This can be caused by several factors such as RFI, however the majority of random flux patterns are caused by the MC3470 amplifier IC within the Disk II Drive itself and is normal behavior within the MC3470.

The MC3470 takes the raw flux pattern signals from the drive head and amplifies it to the 5v TTL logic standard needed for the Disk II Analog card to process and send to the Apple Disk II Controller card. This amplifier IC however is far from perfect in its task. When a disk is being read the amplifier starts out waiting for a change in flux signals under the drive head. If it does not detect a change for a set period it will automatically turn up the signal gain and the process starts again. This will repeat until the MC3470 has turned the signal gain up so much that only random noise is being sent from the IC. This is the actual source of the random bits. This is inherent to the design of the MC3470 and Motorola had redesigned the IC multiple times during the IC's active life to address this noise issue and improve reliability. Some early date codes of the IC are very noisy. Later date codes tend to be less noisy. This is most evident in image files created using the EDD card. The actual noise is also captured during reads and encoded into the image files. When two EDD files are compared they are never the same, and this is due to the amplifier noise added to the image file.

This "noise issue" was taken into account when producing copy protection schemes for the Apple II as the noise would always be dynamic and change locations when being read. A check could be incorporated into a copy protection scheme to read a "blank", unformatted, or specially formatted track to see if the flux patterns were if fact random. If they always changed locations during every read the disk was original. If however these flux bits did not change location and were static this could only happen if the disk was a copy. So now you're starting to wonder "Can I copy a "blank" track?" No, it is impossible on-platform to reproduce a sector or track on a disk which is all north, south, or a degaussed (equal parts north and south) fluxes. This is a limit of the Disk II design and people who wrote copy protections knew this. The disk manufacturers required special hardware, like Applesauce, in order to create these kinds of protected disks. Special drives however are not needed. This is the sole reason some program titles can not be copied using current methods or devices like the EDD. Also lacking in current disk image support is a way to mark a track as random noise and a way for emulators, both hardware and software based, or image to disk programs to reproduce it. This is what the new Applesauce image format will also address.

There has been a rumor over the years that it is impossible to have two zero bits in a row when writing to the Disk II drive. This is correct due to the Disk II Controller board design. It however is not impossible to write disk tracks of all north or all south fluxes off-platform. The amount of noise detected by the MC3870 will be extreme, however, and the Disk II Controller board will have no idea what is going on location-wise with the disk medium. More than two zero bits have no usefulness except in protection schemes.

Considering how noisy the MC3470 can be it's amazing how resilient the Disk II Controller design is and how few issues Apple II users actually experience. We have all experienced the odd issue when writing to a disk, and this is probably related to the slight role the random noise can play.

As you can now see the more correct name for these kinds of data bits or phantom flux transitions and their behavior should actually be "random bits" or "random transitions".

Applesauce: The Hardware

At the heart of the Applesause Controller board is a Teensy v3.2 microcontroller (MCU) module. John has used this visceral and low cost module to act as a Disk II Controller card that can be controlled using a USB connection.

The MCU operates using programmable firmware, or a program telling the IOs on the module what to do and when to do them. It also allows precise control of the Disk II stepper motor and the ability to access all possible tracks from the disk including half tracks and quarter tracks; is responsible for reads, writes, and some very complex calculations related to how the different formats of the disk can be constructed.

It does everything the Disk II Controller card and Apple II do but many times faster and under our direct control. Nothing can be hidden from Applesauce. This kind of control is not possible on-platform and why the Disk II Drive was removed from the Apple II.

Extra processing speed allows for a more granular resolution of 125ns for reading the flux patterns from the floppy disk. This means more precise analysis of the data is possible.

An important aspect of the extra speed is that it allows for oversampling of all reads when needed. This, along with the Applesauce software will ensure that all disks are imaged without errors every time - unlike all other solutions. Should a read be ambiguous and the software can not determine the start of the track or its length, then additional reads are taken for analysis. The user will also be alerted to this possible issue for final oversight and control. This feature also means Applesauce will be able to help restore or save damaged disks in some cases where the disk is just starting to become unreadable.

Users can painlessly update the firmware on the MCU so that they won't be left behind as new features are added to the Applesauce project. It has been designed with simplicity and longevity in mind.

The Applesauce communications protocol will also be open sourced, which will allow developers and end users to write their own software and control it. Users will be able to use and control Applesauce from ANY platform which has a USB port. This allows the project to be widely adaptable and future-proofed as well.

An Applesauce Developer's Kit will be offered consisting of the image specs, an example disk of all applicable copy protections, and explanations on how all the different features will need to work in order to fully utilize the Applesauce hardware. The example disk will allow developers to work with Applesauce and verify their code is working correctly without the need to source a large amount of original software titles.

Applesauce Versions

Currently Applesauce is in pre-release. Only Proof Of Concept, Alpha, and Beta units haven been produced. Proof Of Concept date range was about March till May 2017. Alpha date range was May 2017 till March 2018. Beta date range started March 10th.

As of this date a few testers have received Alpha units. Beta units mark a few major changes, mainly a new microcontroller and power supply, which are currently being fully tested before moving to a final layout and release version.

The Proof Of Concept design mainly revolved around proving the concept of off platform Disk II control was possible via USB, and selecting the best microcontroller for the job. Once there was a direction John stared to layout an Alpha PCB and start a user interface (UI) design. After broad and in depth field testing the lessons learned were added to a refined UI and new layout in a Beta design. Beta was also the first time the Applesauce project received an enclosure.

Applesauce Design Revisions
Applesauce Proof Of Concept from 2017-03-04
Applesauce Proof Of Concept from 2017-03-12
Applesauce Alpha from 2017-05-25
Applesauce Beta from 2018-03-10

Supported Floppy Drives

Disk II, UniDisk 5.25", and UniDisk 3.5" drives will be able to be connected and used with Applesauce as well as all compatible clone drives. The only real limit will be with 5.25" drives and the amount of space inside the drive chassis for mounting the sync sensor which is required for reading some protection schemes.

The most simple drive to use is the Disk II drive. This allows for the simplest access for head cleaning, and they are the most common Apple II drive available.

On February 25th, 2018 John Morris announced UniDisk 3.5" support will be added via a software update to the Applesauce UI. No hardware updates or changes will be required. This concept allows John to continue working on the more used 5.25" disk support and release the Applesauce project sooner. Later, he can return to add the support for the much less used protected 3.5" disks.

Sync Sensor

A major aspect of the Applesauce project will be how to determine where the disk is in relation to the drive head. This is called drive or media synchronization, synchronization for short, and there is no mechanism built in to the Disk II drive to detect or track it. This was done by design and helped save costs, although it made Wozniak's design time a lot longer and more complex. It also costs storage space since track and sector locations must be encoded and stored on the floppy medium. The youtube channel Computerphile has a good video explanation about this (and other great retro videos too).

Apple 2 Floppy Disk Codes - Computerphile

Some copy protection schemes relied on this feature being absent from the Disk II Drive. Along with random bits, this lack of synchronization means disks could be made off-platform and there would be zero chance the disk could be copied on-platform. This doesn't mean it couldn't be cracked; however, it would be unlikely developers would go to such lengths just to allow simple code edits to bypass security. This had been confirmed by Rebecca Heineman who stated The Bard's Tale III: Thief of Fate, although cracked, has never had all the built in security checks defeated (*See here for update). 4AM, a modern and very active Apple II software cracker, has also discovered a lot of titles he/she has cracked also have secondary checks or more. These are usually reported by the user and not found by only looking at code. This is an active issue with the cracking method.

In order to overcome this lack of synchronization issue John designed a sync sensor with help from Henry and added it to the arbor of the Disk II drive. It connects using a cable to the Applesauce mainboard. Disk locations are tracked by the MCU and the result is Applesauce can very precisely locate a flux transition on the disk medium. This combined with the oversampling means Applesauce is able to guarantee a disk is read correctly every time. This is not possible on-platform or with any other copy related program or hardware.

It's very simple to install the sync sensor, and it can be installed in the Disk II and UniDisk 5.25" dives. In contrast, UniDisk 3.5 already haa a built in sync sensor. John will release an install guide so users can perform the installation themselves. ReActiveMicro will also offer the service, as well as sell drives already tested and upgraded.

To install the sync sensor the user just needs to open the drive, attach a small piece of magnetic tape to the hub arbor, then attach the sync sensor close to the drive chassis with some sticky tape. The cable is run out the rear of the drive and connects with the Applesauce board. The UI has built in tests and helps the user adjust and set the sensor to the most reliable position. Opening the drive also gives the user a chance to clean the drive and adjust the speed. Most high use users will want to use a Disk II drive and leave the case open as it's much simpler to clean the heads when needed. The whole install and testing process takes about 10 minutes and is by no means difficult in any of the steps.

Sync Sensor On Drive
Example Of Sync Sensor Installed On Disk II Drive.
Closer Example Of Sync Sensor Installed On Disk II Drive.

In contrast, here is the first video in a set of three which run for about 35 minutes in total on how to modify a rare floppy drive used with Kryoflux, a leading floppy preservation solution. There really should be a warning with this video series which states "Do Not Try This At Home". Besides the near impossible task of finding the rare drive, you need what seems like a decent set of tools or access to a machine shop and the required skill to operate things like dremels and drill presses. There is grinding of the drive chassis, Track Zero sensor arm, and even the head carriage - we joke not. And all of which could later damage the drive, leave it unoperational, or even worse - produce sharp objects within the drive enclosure which could enter the floppy disk and scratch it. A near full disassembly of the drive is also needed as well. And once these mods are done the user is then required to add a transistor, cut some traces, and do some soldering of flywires.

KryoFlux Tutorial: Modify 5.25" drive to read "flippy" disks in one pass / Part 1 of 3

Applesauce: The Software

The User Interface (UI) for Applesauce is GUI based, and is only supported on the Mac (macOS 10.11) at this time. There are plans to have it ported to X86, Raspberry Pi / Linux, and possibly some older macOS versions.

Users will be able to use the UI to painlessly update the firmware on the MUC so that they won't be left behind as new features are added to the Applesauce project. It has been designed with simplicity and longevity in mind.

All of the disk imaging parts of the UI are completed and working well. Imaging unprotected disks takes about 11 seconds. Imaging protected disks takes about 2-3 minutes and will depend on what issues or errors are discovered on the disk. Clean disks will image faster than ones with mould or bad sectors for example. A major feature of the UI is the ability to help salvage damaged disks. Some disks will be able to be saved by utilising oversampling. The UI will then take the best of 5 reads and save what data matches consistently. It will then ask the user for direction for the other sectors. The best part is what data is salvaged will not be thrown out if part of a read is bad. All known good data is stored and new good data can be added later. So users with several bad disks of the same title and version should in theory be able to salvage the disk image by reading all the known good areas of the disks, which Applesauce will then compile into an image until the image is fully built. You'll also be able to open up a different disk image formats (A2R, EDD, NIB, etc) of the same software title and use it to patch the bad data.

Another major part of the UI is the disk analysis/editor which is similar in feel and use to the old nibble editors. The UI will process the data read and show you the nibbles after the analysis tools have annotated the data. Users will be presented with color-coded hex views that show you the function of every nibble and every hidden sync bit. In most cases it will also point out where things like the copy protection exists and the techniques they are using to accomplish it. It gives you the ability to go high tech "digital spelunking" through the disk image in ways that the the early cracker could have only dreamed of.

Another great aspect of the UI will be the ability to test floppy drives to see how noisy the MC3470 is. The advantage to a less noisy drive would be to more easily allow Applesauce to identify small random bit areas. This is also a good diagnostic tool to check the drive and Applesauce hardware.

The "2017-12-09 - Applesauce UI" pic shows recent advancements for those looking to get very technical. John added a "waveform area" to the display. This shows the direct output from the MC3470 which are 1uS (microsecond) high pulses against the vertical lines which are spaced every 4uS, and above that are the nibbles. This output should not be confused with the actual flux transition on the magnetic medium. The graph is read from left to right. The blue waveforms are the 5 oversampled reads performed by Applesauce. You can see they each vary slightly on the right side due to the fluctuations of the drive motor speed. Nothing is perfect, and this is a perfect example of that. The top green waveform is output after Applesauce is done with analysis and is the best interpretation of what was read. Users will also be able to zoom in and out of this area.

This waveform analysis will be very useful for determining random bit areas and where it resyncs, and also for identifying read errors. Random bits would show up as totally different data for each pass performed. Read errors would show up as a missing high transition or an extra one added to the graph and the nibble value displayed will be affected and change. With read errors Applesauce will do its best to recommend what should be in the damaged area however users will ultimately have final input and be able to reread or manually edit what the nibble should be. This powerful feature will make restoration work extremely simple for end users. For crackers this powerful feature allows them to more easily identify protections schemes unlike never before. Users are now about to see the bit patterns and nibble value in one simple UI.

The "2017-12-09 - Applesauce UI" pic shows the image for Ultimate V. Outlined are the blue areas with the little number below. These show if that nibble has timing bits (zeros) following it. If a nibble is a standard 8-bit nibble, then no number is shown. This should be the case for all normal nibbles outside of sync ones. However protections often like to throw in extra timing bits to confuse nibble copiers. For example they used a nibble which has three zeros in a row - which is impossible to do on platform and impossible to copy. The red bits show this kind of protection was detected. A programmer would then have their protection check if this nibble is correct, and if the disk was copied on platform then it will fail the check and can be easily identified as a copy. The red color of the nibble means that Applesauce noticed an issue with the construction of the nibble. This can also be caused by flux timing that is unusual (not clocked properly), or that the resulting nibble value is invalid. In this case, F1 is not a valid nibble value since it has 3 zeros in a row. So, the value can exist, but it is potentially unreliable due to the MC3470 possibly freaking out and throwing a fake pulse.

On 2017-12-10 John announced his work on the Applesauce UI to include several new features which are MC3470 Noise Analysis and Drive Calibration Tools.

Applesauce UI - MC3470 Noise Analysis
Applesauce UI - Drive Calibration Tools

The MC3470 Noise Analysis allows users to test their Disk II Drives to see how "noisy" the MC3470 amplifier IC is. The noise is what is responsible for random bits being produced but could also lead to more errors when reading a disk. If the MC3470 doesn't see a transition after a set time it turns up the gain and the timing loop is repeated. In situations when no flux transitions happen for a long time on the media, as when no disk is inserted in the drive, the gain becomes so high that background RF noise and normal fluctuations inside the IC start to become detected and get output as data. Less noise is better in most cases, however without any noise the protection schemes that rely on random bits will not work. Too much noise could lead to unnecessary user intervention to complete a backup image.

For the MC3470 Noise Analysis, each graph is 16 seconds of sampling. And the final score is the average number of random bits thrown in a single rotation of the disk (200 milliseconds). It has yet to be determined which is the "sweet spot" for random bits, however we will update this page with the information. John noticed while doing R&D that the noise wasn't really random at all, but was actually pretty consistent in the number of random bits it was throwing per second. He then discovered that the quantity of noise associated with a particular chip follows that chip when it is put into another drive. If new MC3470 amplifier ICs become in demand due to Applesauce then be sure to look for ReActiveMicro to offer them.

Drive Calibration tools will allow the user to quickly and simply adjust their drive speed off platform. This is a first for Apple II. Also planned as part of this screen is drive alignment. Users will be able to check how well aligned their drive is which can also help reduce read errors when creating images.

December 2017, an Applesauce was sent to an avid Apple II software collector with a huge collection of software. To date they are the only beta tester willing to spend the hours of time needed to image software and test the UI in depth over the period of weeks and report back to John about feedback, issues, and UI improvements. They tried to use an EDD+ card to image software however they ran in to an alarming failure rate and the majority of the images were useless. Applesauce however has helped identify defective disks, bad sectors, and allowed for imaging of all software. John has used the images and feedback to put the finishing touches in to the Applesauce UI.

Applesauce UI - Visual Display Of Degaussed Disk Damage
Applesauce UI - Closeup Of Degaussed Disk Damage
Applesauce UI - Example Of Bad Nibbles Identified

As of mid-January 2018 John added UI support for EDD files. This allows users to load and edit/examine EDD images, unlike all other solutions. It shows tracks and other such image features. More news to be announced about this newest feature.

Here are some more image examples from Applesauce. These images are produced by the Applesauce UI while the disk is being imaged, and represent the actual floppy medium. They will allow for "fingerprinting" and checksumming of images. It will also allow a simple way to identify "hidden" or nonpublished versions in programming or protections. All other image formats or preservation solutions do not allow for this.

Applesauce UI - A Spiral Disk Example
Applesauce UI - A Dirty Floppy Disk Or Head Example

The "Dirty Floppy Disk Or Head" pic shows another example of the usefulness for the floppy image. Users will be able to tell just by looking at the image on the screen if their head or disk is dirty. You will notice the areas in the red circle. This area is not common to all the rest of the image data. These "ghost" type patterns tell us there is probably some dirt on the disk in these areas, or possibly the medium is bad and damaged. Mold is common on floppies, as is the actual magnetic coating flaking off of some lower quality disks. In this case the user was able to clean the disk and produce a good image.

Applesauce Image File Format (.WOZ and .A2R)

One of the main things that Applesauce is bringing to the table is new disk image formats.

The initial disk image format design is completed and work has started with emulator developers to start integrating support for the new formats. The new image formats have been well received by folks as it addresses all of the shortcomings of current disk image formats like DSK and NIB. Talks have also started with some hardware developers to support the new formats within their virtual floppy drive projects. This will allow still copy protected disk images to run on real hardware.

The original proof of concept image file extension was ".A2D". However at KFEST 2017 Jason Scott made a request of John that the extension be changed to ".WOZ" as an homage to Wozniak.

The .WOZ format however is only the track, sector, and data information. There is no timing information. Other existing image formats will be able to be converted to .WOZ. .DSK images for example need to be nibbilized and converted back to a floppy structure format. The best features of the new format however is it allows for less overhead requirements for hardware or software emulators, as well as room for meta data such as name of the program. It will also allow for markers to be used to track unformatted tracks from random bits areas which will allow designers to know when to produce random output. The size of a .WOZ file is about 250k max and can be much smaller if large areas of the disk are unused.

On January 15th, 2017 John Morris released the .WOZ file specifications for the new format called WOZ Disk Image Reference v0.91. This was a preliminary release of the file spec and the Apple II Community was asked for their feedback. The document goes in to great detail how to use new file format with emulators or custom software.

On January 17, 2018 John Morris released WOZ Disk Image Reference v1.0. The link however is to the final release version which is considered the latest and most current.

Applesauce WOZ Disk Image Reference v0.91
Applesauce WOZ Disk Image Reference v1.0

In comparison .A2R, or the raw data format, contains all the data on the disk including timing. All flux transitions read for the disk are saved in this file. The size of a file is about 20Megs. The file specs has not been fully finalized, published, or released yet. It also may not be released since it's proprietary to Applesauce and leaves John Morris a lot more freedom to modify things as he sees fit rather then be locked in to one set of rules.

Hardware And Software Emulators

The main issue with the new file formats will be their initial lack of support. If nothing uses them, what good are they besides preservation?

John has been in contact with as many designers and programmers as he could to discuss working with them about adding .WOZ support to their hardware and software. Several software emulator developers have started enhancements as soon as .WOZ files spec and sample files were released. With some luck and hard work official announcements will soon be made about their support for .WOZ and start to distribute beta versions of their programs.

In early May John added .WOZ support to OpenEmulator for the Mac. This was a major first step in helping him prove the new file format was working. John reported it took him only a few hours to cleanup the existing code and achieve something that was working. This sparked hope other authors would be able to implement .WOZ support as simply if they decided to do so.

On May 21st, 2018 Gerard Putter announced his "Virtual II" emulator for the Mac now has .WOZ support. This marks the first software emulator without John's major involvement to support the new format.

Hardware designers however have had more of a challenge to meet the requirements. All current hardware emulation platforms do not support .WOZ. Not the CFFA, Floppy Emu, SD Smart Drive, SDFloppy II, SDDisk][ Plus, or UniDisk Air. There is some hope the CFFA will be able however Rich Dreher has not commented about any planned support. And with some existing bugs left unfixed and his public statement about this being the last run of cards using the current hardware it is not likely support will be added in the near future - if at all until a revised project is attempted at some unknown point in the future. Steve Chamberlin commented that the current Floppy Emu can not support the new .WOZ format due to the limits of the current Microcontroller. However Steve did mention he is planning to revise the Floppy Emu at some point and will plan to support the new format then. All other designers have expressed no interest in reworking their projects to add support and most are in "End of Life" status anyway.

On June 20th Kboo HK shared an update about his new emulator project and his success in adding in .WOZ support. This marks the first time a hardware solution has been successful.

Those who purchase older floppy emulation solutions should be aware of these limits and understand that hope in running protected software and all newly imaged programs will not be possible. If this changes we will be sure to update this page with links to announcements.

Crack Versus Copy

Two solution exist which would allow a program to be fully copied and usable. One is the difficult "crack" method and the other is the simpler "copy" method.

Cracking a program is basically circumventing any protections built in to it. The protections could be added for multiple reasons however are usually based on enforcing payment for use. Cracking can range from using a copy program, to "digital spelunking" through the entire program, and there were also Copy Boards like the Wildcard which aided users in tracing code and manipulating the CPU.

The process to completely crack a program ranges from the very simple to the almost impossible. First the person who is doing the cracking will usually need some moderate to deep understanding of the platform and CPU. This enables them to follow lines of code and hopefully identify all protection schemes and bypass or disable them. The issue with this however is various protection routines can be hidden almost anywhere. There is rarely just one protection check, for example. And the protectors have a huge arsenal and bag of tricks at their disposal. Among these are being able to use random flux bits and lack of drive synchronization. Programmers and protection companies would often have layers of protections, checks that checked the checks to see if they were disabled or altered, and even ways to disable the program to make it unuseable, corrupt output, or unwinnable. A cracker may believe that have found all the protections and can start to play a game, however only if you play it against an original copy would you ever notice the difference. To really ensure all protections have been removed would entail going through all lines of code that make up a program and fully following them and understanding what they do and why. And the most obvious issue is a cracked version IS NOT an original. The art of the protection is lost and future wanna-be crackers will never be able to investigate the design.

Copying a program is usually much more simple however the random bits and the lack of synchronization means disks could be made off-platform and there would be zero chance the disk could be copied on-platform. This doesn't mean it couldn't be cracked, however it would be unlikely developers would go to such lengths just to allow simple code edits to bypass security. This had been confirmed by Rebecca Heineman who stated The Bard's Tale III: Thief of Fate, although cracked, has never had all the built in security checks defeated.

Although copying is the preferred method when archiving compared to cracking, it also has its own pitfalls and can be inconsistent even at the best of time. With copies we are still tied to physical media and drives. Applesauce will finally break this dependence AND allow us to have 100% exact copies with protection intact.

Applesauce At KFEST 2017

John K. Morris's Presentation At KFEST 2017

imfEDD Software No Longer Being Updated

Antoine Vignau has announced that imfEDDup will no longer be updated directly once Applesauce is available due to the superiority and completeness of Applesauce IMAGING compared to EDD when EDD is being used for IMAGING only. The EDD board STILL has its purpose, which was its original intended purpose which is to do DISK to DISK copies only which is a very good "cost effective" way to backup your media. The EDD can still image your disks, but there are some caveats that should be know. Also, if there is no copy protection on a disk and you are using a EDD to make an IMAGE and not Disk to Disk, then you should have no issues with EDD and the imfEDDup software. If you do decide to use imFEDDup and the EDD card to make images of a copy protected disk, be sure to test your images with Passport software as well as run the software to be sure that it is in fact a good image.

Imaging with imfEDDup and the EDD card has drawbacks:

There is no oversampling.*
The EDD card also captures all drive "noise" when it makes an image, so each image is always different. We can never "fingerprint" a disk (MD5 for example).
EDD images can not be cross compared to one another - even made from the same original floppy on the same day as both will always be different and even different sizes.
EDD is unable to report or detect if there is physical damage or degaussed areas of a disk.
Due to the above issues how do we know if one program is a different version from another if it's not explicitly stated? There could be multiple versions of protections, or code, or both and we would never know with EDD images.
EDD can't successfully copy software which uses protections based on track synchronization or random bits for example. Other protections also which rely on timing.
The EDD card through software calculates the timing bits to know if a nibble is 32, 36 or 40us since it can not obtain this information directly from the medium, controller, or drive. This info is not captured from the floppy medium itself.

*imfEDDup makes multiple copies of every track.  If a sector has an error, there is always another chance that it will be captured correctly however these two reads are not and can not be compared before writting to the image. You can determine good vs bad reads via the sector checksum, but it isn't perfect because it is only an XOR checksum and it is very easy to have errors that will still calculate out to the same value. If a track isn't sector-based, then there is really no way to determine if the data is good or bad.

KryoFlux Limitations

KryoFlux began back in about 2010. The concept was to allow users to more simply preserve software, which was not a new concept. Catweasel was a similar project from 1996. Catweasel however had some limitation which it seems KryoFlux attempted to address such as larger platform support and availability.

As with the EDD, KryoFlux also suffers from many issues when dealing with the Apple II platform. The main issue is the Apple II's CPU controls the floppy drive, which allows for some very complex protection schemes unlike all other systems. Several users have reported the following issues when using KryoFlux related to the Apple II platform (and several others as well):

Can’t really write to a floppy, and doesn’t read the best either.
Super picky about the floppy drives you use.
Some floppy drives require they be sent to KryoFlux to be modded before use.
Can’t read double sided floppies without a specific model drive which needs to be modified.
The only focus is on archiving and not duplication.
The archive format doesn’t work on much of anything and has no support with emulators.
Primary UI software platform is Windows, and crashes a lot.
As with Apple II, there is very poor support for Atari.
Their software is very closed source. No spec is available for writing your own software to drive the KryoFlux or use the existing image format.

Besides all the technical issues and shortcomings, most disheartening is KryoFlux Preservation Technology Group (KFPTG) performs a background check on customers before shipping an order. It has been reported they will not sell units to companies, people who could be perceived as possible competition, or places in which they believe more money could be made performing services. They don't sell to museums or schools for example. They ONLY will provide services. Their business model is one of offering their services to help archive your floppies more than hardware design, innovation, community support, or allowing a simple, trouble free, self-service solution.

In comparison Applesauce addresses all of the limitations of the Kryofux listed above. The core concept is to allow a simple, trouble free, self-service solution, as well as to support the retro community with new and innovative image format standards to allow broad use. Several emulator developers have been contacted to help support the new .WOZ format. This will allow a much better user experience.

Preserved Software

March 10th, 2018 John Morris let ReActiveMicro know that close to 3000 disk sides have been imaged. Two main beta testers / collectors are responsible for this work over the past couple months and their estimated donation will probably break 3500 sides. This still accounts for only a small percentage of Apple II software. However some very rare titles have been preserved, and many more still need to be preserved.

January 20th, 2018 John Morris announced "There are already over 1000 disks (primarily games) that have been imaged with Applesauce. ;)

December 28th, 2017 Antoine Vignau announced a couple very early and rare games on floppy disk for the Apple II were imaged, preserved, and released all due to Applesauce.
Paddle Fun (1979) and Skybombers II (1980).

December 2017, an Applesauce was sent to an avid Apple II software collector with a huge collection of software. They tried to use an EDD+ card to image software however they ran in to an alarming failure rate and the majority of the images were useless. Applesauce however has helped identify defective disks, bad sectors, and allowed for imaging of all software. John has used the images and feedback to put the finishing touches in to the Applesauce UI.

July 31st, 2017 John Morris announced Bake And Taste was imaged and sent to 4AM for cracking. This marks the first announced program to be distributed due to Applesauce.