In porting the CHDK to the Canon IXUS 870 IS, I encountered a common problem: lack of (good) documentation. An enormous amount of knowledge floats around on the wiki, forum and (probably mostly) inside the heads of the developers, but finding the right information is no easy task. In an attempt to spare future porters a decent dose of frustration, I have attempted to document what I have unearthed
during my port, mostly with respect to getting the bigger picture. Many thanks are required for the assistance from the people in the CHDK IRC channel (especially reyalP).
Please note that I'm not blaming anyone (although I possibly should); I very well understand that writing documentation is not the most interesting activity. To be honest, I'm also not really known for my great documentation.
Also note that the documentation below is meant to be put on the CHDK wiki (in some form or another) such that others can easily correct and extend it. It is probably wise to check there first for a possibly more up-to-date version.
Requirements
Besides some knowledge of assembly and C, you will need to following things when making a CHDK port:- A dump of the Canon firmware (details)
- Tools to disassemble the above firmware (e.g. IDA or others)
- CHDK source + ARM compilation tools (see links here)
- Some addresses for LEDs (discussed here)
The workings of the CHDK
To understand how the CHDK works we first must look at the Canon firmware itself. On startup, the firmware roughly executes the following:
- If there is a memory card with a DISKBOOT.BIN file in the root: load, decode and run it.
- Otherwise, start the normal booting process, which includes starting various tasks such as:
- Logging task
- Keyboard task
- Image-capturing task
- File-system task
The CHDK adds another task and slightly adapts some of the existing task to make things work. It does this mimicking the normal boot process, substituting different behaviour where needed. This includes adding hooks to the general task creation of the firmware, which allows it to start its own versions of certain tasks.
All of this is possible due to the DISKBOOT.BIN execution. Put the CHDK code in DISKBOOT.BIN, and the firmware will execute it automatically. Note that this requires a FAT12 or FAT16 partition (details). Also note that it is also possible to do the same by using a PS.FIR/PS.FI2 file and manually choosing to do a "firmware update".
The details of what happens when DISKBOOT.BIN is executed are as follows.
- The core CHDK code is copied to a suitable location in memory.
- The camera is restarted with the CHDK code as new entry point (based on reset code from the firmware).
- The CHDK adaptation of the boot process is run. It has the following differences from the firmware boot process:
- It adds task creation hooks.
- It starts the main CHDK task.
- Changes some details to ensure correct startup
- At the end of the above process, all (normal) tasks are created. Via the added hooks, the CHDK substitutes some slightly modified tasks.
Making a port
To make a port for a new camera one has to search the firmware for code to base the adapted boot process and tasks on. Also, various camera-specific details and functions that the CDHK uses need to be implemented. The latter mostly entails taking already existing code of a port of a similar camera (i.e. your reference port) and finding the right constants for that code in the firmware. You typically find the origin of values/code in the reference port's firmware and try to use that to find something similar in your own. Finding a good reference port might not be trivial, so you might want to consult the CHDK experts on that one.
The easiest approach is probably to take all the code of an already existing port as basis and adapt it file by file. You'll need copies of the loader/<camera> and platform/<camera> directories, add(/copy) defines in include/camera.h and add the firmware dump to platform/<camera>/sub/<version> (N.B.: change the directory in sub to the right name for your firmware version).
Next you make sure that you can compile the CHDK image. Add defines for PLATFORM and PLATFORMSUB to makefile.inc (commenting out the ones that are enabled by default) and run make fir. There might be errors due to undefined functions. For now, you can get rid of them by adding some dummy values to platform/<camera>/sub/<version>/stubs_entry_2.S (e.g. NHSTUB(name,0x12345678) for function _name; note the underscore which is not in the NHSTUB).
Now we can start adapting the code. During this process one can add debug statements that blink LEDs to indicate what the camera is doing. For example, I used something like the following code (with 0x12345678 the address of the LED and N a reasonably large number such as 0x1000000):
int i;
*((volatile long *) 0x12345678) = 0x46; // Turn on LED
for (i=0; i<N; i++) // Wait a while
{
asm volatile ( "nop\n" );
}
*((volatile long *) 0x12345678) = 0x44; // Turn of LED
for (i=0; i<N; i++) // Wait a while
{
asm volatile ( "nop\n" );
}
Note that you cannot safely insert this code everywhere, so it is probably best to make a function (no inline!) for this and use that instead. Also note that in some cases you might want to add a while (1) ; to prevent not yet adapted code from running.
As we all know, it is good practice to document the code. For the CHDK port this mostly means that you should note where (and, in non-trivial cases, how) you have found certain values or pieces of code in the firmware. Also make sure it is clear which portions of assembly are straight from the firmware and which are changed, added or removed. Besides being useful when debugging, this documentation really helps people who are going to use your port as basis.
I only cover the most important details here. Files or functionality that is not mentioned are typically straightforward to adapt (e.g. Makefile) or similar to other files/functionality discussed here. When you think you have done "everything", just go over each file to see if you have missed anything.
platform/<camera>/sub/<version>/makefile.inc
This is probably the first file you want to adjust. At first the most important values are:
- PLATFORMOS: either vxworks or dryos (it's DryOS if "gaonisoy" is at the start of the firmware dump)
- MEMBASEADDR: typically 0x1900 (for DryOS: the smallest address used in the loop above the occurrence of MEMISOSTART)
- RESTARTSTART: should point to a piece of memory that can be used to copy the restart code to. A safe bet is usually MEMBASEADDR plus the size of DISKBOOT.BIN (take it a bit bigger to accommodate changes in DISKBOOT.BIN), but be sure it does not cause overlap with MEMISOSTART (see loader/<camera>/main.c)
- ROMBASEADDR: should point to the start of the firmware (usually 0xFF810000 or 0xFFC0000, related to size of firmware)
- MEMISOSTART: points to the start of the memory pool used by the firmware (you can find this address at the end of the first piece of code in the firmware for DryOS)
- NEED_ENCODED_DISKBOOT: for new cameras this it seems a safe bet to say "yes"
loader/<camera>/entry.S
The code in this file is the very first thing that gets called when DISKBOOT.BIN (or PS.FI2) is run. It typically just consists of the following code that calls the function in loader/<camera>/main.c:
mov sp, #<MEMBASEADDR> mov r11, #0 b my_restart
Some cameras seem to need some additional code before this piece for the automatic boot to work. See, for example, the SX10 port for the code snippet.
As a test you can already add some debug code to turn on a LED here, but you can also just add the code to the beginning of my_restart() in main.c. If this doesn't work, you should make sure you have the right numbers in platform/<camera>/sub/<version>/makefile.inc.
loader/<camera>/main.c
Here we implement my_restart(). The standard code copies the reset code from the resetcode subdirectory to a safe place (at RESTARTSTART) and then calls that code. The reason it is copied is because when copying the CHDK core to its destination, which happens in the reset code, there might be situations where the destination and source (containing the executing reset code) overlap. Overwriting code that is running is not what we want.
Note that the call to the reset code is a bit of a hack. Instead of directly calling the function copy_and_restart() we actually jump to the entry point of the reset-code binary. That is, we jump to the code in loader/<camera>/resetcode/entry.S. The arguments we pass in the C code are "silently" passed on to the actual copy_and_restart() function.
loader/<camera>/resetcode/entry.S
Similar as the entry.S before, but this one calls the copy_and_restart() from loader/<camera>/resetcode/main.c. Note the "silent" passing of arguments mentioned earlier (../main.c).
loader/<camera>/resetcode/main.c
Here we implement the function copy_and_restart(). As arguments it takes the destination, source and size of the CHDK core. It first copies the core to the destination and then executes the slightly adapted reset code. You can typically locate this reset code in the firmware by using your reference port. Once you have found it, you must copy it (or make sure it is the same as the reference) and make sure you change the jump at the end to jump to the start of the core you just copied).
Note that this last jump is to the start of the core which means it will execute the code in core/entry.S next. This code will call the startup() function in platform/<camera>/main.c, which in turn will call the adapted boot code.
platform/<camera>/sub/<version>/boot.c
In this file the adapted boot process is located. For testing purposes you probably want to make sure that you have the standard startup() in platform/<camera>/main.c; this function basically just calls the boot() function of boot.c.
Besides the boot process itself, this file also contains the task-creation hooks, a function to start the main CHDK ("spy")task and one or two modified task (init_file_modules_task() and, if relevant for your camera, JogDial_task_my()). At first it is probably best to comment out the code of the task-creation hooks and CreateTask_spytask(). This way you can focus on getting the boot process working and uncomment the tasks as you finish them.
The boot code is taken from the start of the firmware and the successive functions that are called from it. There a a number of changes that are made:
- The task-creation hooks are added.
- Power-button detection is improved; this makes sure that you only have to hold the power button for about a second to start in shooting mode. (Due to the more complex booting the firmware no longer correctly detects whether the power-button was pressed to start the camera.)
- MEMISOSTART is replaced; the CHDK core is now load at that address so we must adjust the memory pool accordingly.
- The CHDK task must be created.
In essence you can just take the code from the firmware and copy the changes as made in your reference port. Once you have got the boot code ready, the camera should boot normally again. (Only difference is probably that a short press on the power-button gives you the review mode now.)
First thing to do now is to get the CHDK task to run. So uncomment the previously commented code and check that you get the CHDK boot screen now. It will take a few seconds because it is waiting for something that has to be done at a later stage. The relevant code of the CHDK task is in core/main.c. Also, you might need to fix some of the references in stubs_entry_2.S and stubs_min.S as well as the vid_*() functions in platform/<camera>/sub/<version>/lib.c and debug_led() in platform/<camera>/lib.c.
At this point you can probably skip the tasks in this file. It is more useful to first get the keyboard to work properly for CHDK. (See platform/<camera>/kbd.c.) Don't forget to uncomment the if statement with mykbd_task!
Following the general keyboard task, you can add the JogDial_task (if your camera has a jog dial). Here you will have to add a piece of code (see, for example, the SX10 port). The purpose of this code is to be able to stop the firmware from reacting to the jog dial while one is in CHDK ALT mode.
Finally, you must adapt init_file_modules_task(). Here an extra piece of code has to be added as well. This code is to support the autoboot feature on memory cards that are bigger than 4GB (by using two partitions). Also, after the call that takes care of this, we add a statement that signals the CHDK task it can start. This is done afterwards to make sure that this task uses the right partition.
platform/<camera>/sub/<version>/stubs_entry_2.S
The CHDK uses various functions that are readily available in the firmware. In order to do so the addresses to these function need to be known. The build system tries to guess (details) these addresses from the firmware dump (which are written to platform/<camera>/sub/<version>/stubs_entry.S), but not in all cases does it find the right addresses. For this reason it is possible to add corrections to stubs_entry_2.S. These corrections should be of the form NHSTUB(name,address). Note that the macro NSTUB, as used in stubs_entry.S, should not be used; using NHSTUB ensures that the incorrect addresses found automatically are overridden.
There might also be some functions that are completely missed by the automatic detection. These will generate compilation errors that mention function names that start with an underscore. These should also be manually added to stubs_entry_2.S, but without the underscore (NHSTUB takes care of that).
platform/<camera>/kbd.c
To get CHDK to detect button presses (and USB remote triggers), one needs to specify the bits that represent these. These bits are located in the physw_status array (defined in platform/<camera>/sub/<version>/stubs_min.S). To figure out which bit is what, one can use the debug On-Screen Display (OSD) of CHDK. As the buttons probably don't work yet, you can force it to be always visible by uncommenting the commented call to gui_draw_debug_vals_osd() in core/gui.c and removing the if at the beginning of that function. Also make sure that in this function you display the physw_status array (one line for each of the three elements). With these changes you should be able to find each bit that corresponds to a specific button. Note that for the USB-power bit you can simple plug in a (connected) USB cable.
The information obtained in this way should be put in keymap[] (except for the USB-power bit). Each entry is of the form { idx, KEY_NAME, mask } where idx is the index to physw_status and mask selects the bit (or bits) for the KEY_NAME button. That is, (physw_status[idx] & mask) should be true if, and only if, button KEY_NAME is not pressed (not, because the bits are 0 if the button is pressed). The table should be ended with { 0, 0, 0 }.
For the USB power you should define the macros USB_MASK and USB_REG (where the latter is the index for physw_status). Be sure to check the use of USB_REG as some code simple has the value hard coded instead of using the macro.
Next you can define alt_mode_key_mask and the KEY_MASK? macros. For alt_mode_key_mask you should take the mask from keymap[] that corresponds to the button that should be used to enable the CHDK "ALT" mode. The macros should be the or of all the masks of key that correspond to the specific index (i.e. for KEY_MASK0 you should take the masks of keys that are represented in physw_status[0]).
Besides these defines, you should also set SD_READONLY_FLAG. This is needed because autoboot only works if your memory card is locked and saving pictures only works if it is unlocked. Because the card lock is not a "true" lock, CHDK can fake that the card is unlocked by unsetting this flag. Note that there typically is no define to indicate the index of physw_status that is used, so make sure that the code you have uses the right index.
Finally, if your camera has a jog dial, kbd.c should implement the functions Get_JogDial() and get_jogdial_direction(). Don't forget to check that turning the dial in, for example, the CHDK menu feels intuitive.
With this file complete, you should be able to actually use the CHDK. Of course only as far as the functionality that you have enabled at this point allows you.
platform/<camera>/sub/<version>/capt_seq.c
In capt_seq.c we have two modified tasks: capt_seq_task() and exp_drv_task(). These are adapted to support saving RAW files, using a remote (USB) trigger and changing the exposure settings (including the addition of very long exposure times).
platform/<camera>/sub/<version>/movie_rec.c
Here we have just on task. An adapted movie_record_task() to allow optical zooming during recording as well as changing the movie quality.
platform/<camera>/lib.c
The function ubasic_set_led() is a bit of an odd one as it has no clearly defined behaviour (due to the fact that different cameras have different number and kinds of leds). The best guess seems to be to just try and approximate the behaviour as described in the documentation of the UBASIC set_led function.
platform/<camera>/main.c
In main.c there are essentially three main things to adjust. First of all we have modemap[] (and mode_get()). The way to populate this table is the same as with key_map[] in kbd.c. Use the debug OSD with "Props" to show item PROPCASE_SHOOTING_MODE (see include/propcase1.h or include/propcase2.h, depending on your camera, for the right number) and change modes to see what value is shown. You might need to extend the enumeration in include/platform.h if your camera has new kinds of modes.
Secondly, there is fl_tbl[] (with CF_EFL and zoom_points). Depending on the number of different zoom steps your camera supports, there are two implementations. One has a table with all possible focal lengths (one for each zoom step; see, for example, the IXUS 980 port) while the other has just a small number of focal lengths and calculates the others through interpolation (see, for example, the SX10 port). To get the values for fl_tbl[] make a picture for each zoom step you put in the table and check the EXIF data for the reported focal length. The entries are in micrometers (i.e. mm*1000). To calculate the value for CF_EFL see the comments in the IXUS 980 port. Note that this value is typically multiplied by 1000 or 10000 to allow for accurate calculations using just integers (this factor is eliminated in get_effective_focal_length()).
To conclude this file, we have the get_vbatt_min() and get_vbatt_max() functions. Get the values by observing the voltages that CHDK reports over time (you'll have to change the display of percentage to voltage in the CHDK menu). You might want to take numbers that deviate a bit from the actually observed minimum and maximum, as the decrease over time is not linear and is likely to be higher at the limits.
platform/<camera>/shooting.c
To fill aperture_sizes_table[] you can use the same method as for filling fl_tbl[] in main.c. In case your camera doesn't have an actual diaphragm, you just take pictures at the available zoom steps. Twice if your camera has an ND filter (once with filter and once without; either use the ND-filter option from the CHDK menu, if you have the right values in platform/<camera>/sub/<version>/stubs_entry(_2).S, or find specific modes that make sure the filter is used or not). The entries are of the form { idx, av, string } with idx starting from 9, av the values of PROPCASE_AV and string a string representation of the aperture (the value of which you can get from the EXIF data).
To fill iso_table[] you check PROPCASE_ISO_MODE for the different ISO values the camera allows you to select. Note that Auto gets index 0 and HI gets index -1.
Note that I haven't personally adapted the shutter_speeds_table[], but to fill it you essentially have to try various exposure times and see if there is a difference with others. If not, you know that the used exposure time wasn't really supported.
Finally, you'll have to find the right values for PARAM_FILE_COUNTER and PARAM_EXPOSURE_COUNTER by using the debug OSD. You can easily detect them by taking a picture and noting which value changes. Note that PARAM_FILE_COUNTER is a big number as it actually contains multiple counters (see its usage in platform/generic/shooting.c; taking a picture will add 16 to the number as a whole.
core/kbd.h
Add your camera to the right definition of nTxtbl. This table should correspond with the zoom-step indices used for fl_tbl[] in platform/<camera>/shooting.c.
Getting your first version out
If all went well, you should have a reasonably decent initial port by now. Post it on the forum, add a link to the wiki page for your camera and keep your fingers crossed.