|
|
Post by oleavr on Nov 29, 2021 1:18:15 GMT
Hello,
I've finally managed to arrive at a kernel config that's very close to the vendor's. The resulting uncompressed kernel is only off by 8 KiB in size, and the differences seem to be mostly about them having access to newer drivers.
(Based on the Orange Pi's SDK sources.)
Note that I've only focused on the kernel itself -- I didn't yet review the enabled modules. But with this config and `make modules_prepare`, using Module.symvers copied from the Maxi, I was able to build Realtek's out-of-tree r8152 driver with only a few ARM-specific tweaks. The driver loads and works, with an Ethernet interface popping up when I plug in a matching USB device. (When I started out with a rough config this would fail due to subtle differences in struct layouts.)
Cheers!
Attachments:config (66.25 KB)
|
|
|
|
Post by jj0 on Nov 29, 2021 9:50:02 GMT
Hi! Thanks for sharing, this is great! As it happens I was working on getting various USB2Ethernet modules compiled (and working) for the Maxi as well, and used offsets and sizes of the struct net_device members and functions switched on or off depending on the CONFIG_.... to try to figure out the right config for getting them to load without errors. I used the nanopi SDK and in the end I found out that the using the sun8iw7p1smp_android_defconfig with function tracers disabled let me compile and load the modules OK. But that config has lots of additional things still enabled that probably aren't used by the Maxi kernel so I'll switch to using your config for future stuff. Specifically regarding the r8152 driver, I used this one with references to usb_autopm_get_interface(_async) and usb_autopm_put_interface and usb_ensable_lpm and usb_disable_lpm removed and the USB VENDOR/PID data for my r8152 dongle added. Do you have a link to the Realtek out of source driver? Attachments:r8152.c (90.81 KB)
.config (80.37 KB)
|
|
|
|
Post by oleavr on Nov 30, 2021 0:39:53 GMT
Hi! Thanks for sharing, this is great! As it happens I was working on getting various USB2Ethernet modules compiled (and working) for the Maxi as well, and used offsets and sizes of the struct net_device members and functions switched on or off depending on the CONFIG_.... to try to figure out the right config for getting them to load without errors. I used the nanopi SDK and in the end I found out that the using the sun8iw7p1smp_android_defconfig with function tracers disabled let me compile and load the modules OK. But that config has lots of additional things still enabled that probably aren't used by the Maxi kernel so I'll switch to using your config for future stuff. Specifically regarding the r8152 driver, I used this one with references to usb_autopm_get_interface(_async) and usb_autopm_put_interface and usb_ensable_lpm and usb_disable_lpm removed and the USB VENDOR/PID data for my r8152 dongle added. Do you have a link to the Realtek out of source driver? Ahh, cool! I started out with sun8iw7p1smp_defconfig, and after a while I started noticing that the _android config had options set the correct way, and felt a nagging regret that I'd started with the default config  I basically wrote a script that diffed the built kernel's symbols vs. a snapshot of /proc/kallsyms from the target. First I focused on the symbols that were missing, figuring out which features/drivers were needed to pull them in. Then once I got that down to only unknown symbols, I focused on the reverse direction, turning off features/drivers to reduce the list of "redundant" symbols, while making sure I didn't regress on the missing symbols. This was a bit painful and tedious, i.e. lots of `git grep` and looking at the nearest Kconfig/Makefile when in doubt, and would probably be worth semi-automating. (I did take a stab at using vamos.informatik.uni-erlangen.de/trac/undertaker and their tailor tool, but kept running into bugs in code that hadn't been touched in years, so ended up abandoning that approach.) Thinking ahead though it would be great to binary-diff the two kernels and reverse-engineer the missing patches, and any config constants that might be different, but for now I'm resisting the temptation to go down that rabbit-hole.  (Perhaps RGL ends up sharing their source-code eventually, or somebody manages to get Allwinner to share their latest H3 tree. I'm tempted to try the latter, even if it's probably futile.)
|
|
|
|
Post by jj0 on Nov 30, 2021 9:44:22 GMT
Hi! Thanks for sharing, this is great! As it happens I was working on getting various USB2Ethernet modules compiled (and working) for the Maxi as well, and used offsets and sizes of the struct net_device members and functions switched on or off depending on the CONFIG_.... to try to figure out the right config for getting them to load without errors. I used the nanopi SDK and in the end I found out that the using the sun8iw7p1smp_android_defconfig with function tracers disabled let me compile and load the modules OK. But that config has lots of additional things still enabled that probably aren't used by the Maxi kernel so I'll switch to using your config for future stuff. Specifically regarding the r8152 driver, I used this one with references to usb_autopm_get_interface(_async) and usb_autopm_put_interface and usb_ensable_lpm and usb_disable_lpm removed and the USB VENDOR/PID data for my r8152 dongle added. Do you have a link to the Realtek out of source driver? Ahh, cool! I started out with sun8iw7p1smp_defconfig, and after a while I started noticing that the _android config had options set the correct way, and felt a nagging regret that I'd started with the default config I basically wrote a script that diffed the built kernel's symbols vs. a snapshot of /proc/kallsyms from the target. First I focused on the symbols that were missing, figuring out which features/drivers were needed to pull them in. Then once I got that down to only unknown symbols, I focused on the reverse direction, turning off features/drivers to reduce the list of "redundant" symbols, while making sure I didn't regress on the missing symbols. This was a bit painful and tedious, i.e. lots of `git grep` and looking at the nearest Kconfig/Makefile when in doubt, and would probably be worth semi-automating. (I did take a stab at using vamos.informatik.uni-erlangen.de/trac/undertaker and their tailor tool, but kept running into bugs in code that hadn't been touched in years, so ended up abandoning that approach.) Thinking ahead though it would be great to binary-diff the two kernels and reverse-engineer the missing patches, and any config constants that might be different, but for now I'm resisting the temptation to go down that rabbit-hole. (Perhaps RGL ends up sharing their source-code eventually, or somebody manages to get Allwinner to share their latest H3 tree. I'm tempted to try the latter, even if it's probably futile.)
Yes, it is easy to get sucked down the rabbit hole... Regarding kallsyms, would the attached kallsyms-ko module that publishes all kernel symbols to /proc/kallsyms-ko instead of only the EXPORTed ones be of use for more fine-tuning? Attachments:kallsyms-ko.7z (29.04 KB)
|
|
|
|
Post by spannernick on Nov 30, 2021 14:17:06 GMT
This sounds good, maybe one day we be able to plug any Plug`N`Play network adapter, wired or wireless in and be able to use it with PCUAE and VICE Mode and connect it to a BBS boards... and maybe use another devices too like maybe a printer. We can all wish can't we... |
|
|
|
Post by oleavr on Dec 1, 2021 1:58:54 GMT
Ahh, cool! I started out with sun8iw7p1smp_defconfig, and after a while I started noticing that the _android config had options set the correct way, and felt a nagging regret that I'd started with the default config I basically wrote a script that diffed the built kernel's symbols vs. a snapshot of /proc/kallsyms from the target. First I focused on the symbols that were missing, figuring out which features/drivers were needed to pull them in. Then once I got that down to only unknown symbols, I focused on the reverse direction, turning off features/drivers to reduce the list of "redundant" symbols, while making sure I didn't regress on the missing symbols. This was a bit painful and tedious, i.e. lots of `git grep` and looking at the nearest Kconfig/Makefile when in doubt, and would probably be worth semi-automating. (I did take a stab at using vamos.informatik.uni-erlangen.de/trac/undertaker and their tailor tool, but kept running into bugs in code that hadn't been touched in years, so ended up abandoning that approach.) Thinking ahead though it would be great to binary-diff the two kernels and reverse-engineer the missing patches, and any config constants that might be different, but for now I'm resisting the temptation to go down that rabbit-hole. (Perhaps RGL ends up sharing their source-code eventually, or somebody manages to get Allwinner to share their latest H3 tree. I'm tempted to try the latter, even if it's probably futile.)
Yes, it is easy to get sucked down the rabbit hole... Regarding kallsyms, would the attached kallsyms-ko module that publishes all kernel symbols to /proc/kallsyms-ko instead of only the EXPORTed ones be of use for more fine-tuning? Thanks a lot! I tried it but noticed that symbols belonging to loaded modules were also included. But then it occurred to me that pulling the kernel out of /dev/nanda and turning it into an ELF using github.com/marin-m/vmlinux-to-elf, and finally running `nm` to dump the list of symbols, should in theory be equivalent, without having to filter out the symbols belonging to loaded modules. (I've attached the results with the hacky comparison script used. It's getting late so won't get any further than this today, but thought I'd share it here just in case you happen to have some free cycles ) |
|
|
|
Post by jj0 on Dec 1, 2021 8:23:28 GMT
Yes, it is easy to get sucked down the rabbit hole... Regarding kallsyms, would the attached kallsyms-ko module that publishes all kernel symbols to /proc/kallsyms-ko instead of only the EXPORTed ones be of use for more fine-tuning? Thanks a lot! I tried it but noticed that symbols belonging to loaded modules were also included. But then it occurred to me that pulling the kernel out of /dev/nanda and turning it into an ELF using github.com/marin-m/vmlinux-to-elf, and finally running `nm` to dump the list of symbols, should in theory be equivalent, without having to filter out the symbols belonging to loaded modules. (I've attached the results with the hacky comparison script used. It's getting late so won't get any further than this today, but thought I'd share it here just in case you happen to have some free cycles ) Interesting tool, vmlinux-to-elf. It can probably be modified to dump all symbols as well as the ELF. I'v booted the Maxi without modules (adding RAMFS parameter to the bootargs in uboot) and attached the output of /proc/kallsyms-ko. |
|
|
|
Post by jj0 on Dec 1, 2021 11:31:38 GMT
Thanks a lot! I tried it but noticed that symbols belonging to loaded modules were also included. But then it occurred to me that pulling the kernel out of /dev/nanda and turning it into an ELF using github.com/marin-m/vmlinux-to-elf, and finally running `nm` to dump the list of symbols, should in theory be equivalent, without having to filter out the symbols belonging to loaded modules. (I've attached the results with the hacky comparison script used. It's getting late so won't get any further than this today, but thought I'd share it here just in case you happen to have some free cycles ) Interesting tool, vmlinux-to-elf. It can probably be modified to dump all symbols as well as the ELF. I'v booted the Maxi without modules (adding RAMFS parameter to the bootargs in uboot) and attached the output of /proc/kallsyms-ko. Adding a print statement in elf_symbolizer.py: for symbol in kallsyms_finder.symbols:
print(symbol.name)
prints all symbols. If I compare these to the ones from my RAMFS dump they are essentially the same except for the symbols added by the kallsyms-ko module: cleanup_module
init_module
kallsyms.c
kallsyms.mod.c
__mod_author150
__mod_description151
__mod_license152
__mod_srcversion37
__module_depends
__mod_vermagic5
Our_Proc_File
procfile_read
procfile_write
procfs_buffer
procfs_buffer_index
procfs_buffer_size
prsyms_exit
prsyms_init
prsyms_print_symbol
__this_module
____versions
And a number of $a and $d symbols that can be ignored: On ARM machine, $a/$d symbols are used by the compiler to mark the beginning
of code/data part in code section. These symbols are filtered out when
linking vmlinux(see scripts/kallsyms.c ignored_prefixes), but are left on
modules. So there are $a symbols in /proc/kallsyms whose address overlap
with the actual module symbols and can confuse tools such as perf when
resolving kernel symbols. |
|
|
|
Post by c64stuff on Dec 1, 2021 13:28:06 GMT
You guys are doing amazing work. I bet people will start snatching up any mini stock left on the shelf in the states now (the Maxi sells fast though here) that they know they can do everything from multiple platform emulation to using wifi modems, etc. I expect it won't be long until people are also actually hooking up real commodore disk drives and printers to the mini and Maxi using a USB dongle to interface with them. The ultimate would be somebody actually using a real commodore modem with it to run a bbs and people actually dialing in.
Question: Will any of this progress mean you can mod the way it mates with USB joysticks and gamepads so it's more universal in recognizing them and doesn't require tweaking the config file to try to get them to work?
|
|
|
|
Post by jj0 on Dec 3, 2021 8:34:31 GMT
Question: Will any of this progress mean you can mod the way it mates with USB joysticks and gamepads so it's more universal in recognizing them and doesn't require tweaking the config file to try to get them to work? No, I don't think so. Recognising joysticks is governed by the data in the gamecontrollerdb.txt file and that is not affected by the stuff in this thread. |
|
|
|
Post by c64stuff on Dec 3, 2021 20:32:16 GMT
I thought the issue with the configuration of controllers was that certain options weren't turned on in the RetroGamrs firmware, just like certain ones weren't turned on for vice for things like interfacing with USB for stuff like wifi modems, but now that's possible?
|
|
|
|
Post by spannernick on Dec 4, 2021 11:25:25 GMT
The Carousel uses SDL for its joysticks so thats why you have to use the gamecontrollerdb.txt file, I think your talking about modules used with the kernal and if you can turn things on or off now in the kernal, I do not think we are there just yet. Yes a joystick might need a kernel module to work like the Xbox 360 Controller. but some older USB joysticks or controllers just do not work, I have one here that doesn't work from 2004, USB has been around from 1998 so not every one will work in the machine, its pot luck.... The Original Xbox controllers do not work. |
|
I basically wrote a script that diffed the built kernel's symbols vs. a snapshot of /proc/kallsyms from the target. First I focused on the symbols that were missing, figuring out which features/drivers were needed to pull them in. Then once I got that down to only unknown symbols, I focused on the reverse direction, turning off features/drivers to reduce the list of "redundant" symbols, while making sure I didn't regress on the missing symbols. This was a bit painful and tedious, i.e. lots of `git grep` and looking at the nearest Kconfig/Makefile when in doubt, and would probably be worth semi-automating. (I did take a stab at using 
(Perhaps RGL ends up sharing their source-code eventually, or somebody manages to get Allwinner to share their latest H3 tree. I'm tempted to try the latter, even if it's probably futile.)
