Notes on Linux 5.16 RC1 Deployment for Apple M1 Mac Mini

What works / doesn’t work?

  • 10G Ethernet: Works.
  • WiFi: Can’t find wireless card.
  • Audio: Neither HDMI nor onboard audio works.
  • USB Audio: Works. If you use Apple USB-C to 3.5mm dongle, connect your headphones / speakers to the dongle first before plugging the dongle into Mac Mini.
  • USB3: Degraded speed on type C ports – Operates at 300Mbit/s. Type A ports are fully functional.
  • Graphics: fbcon / kmscon works without issue. Can start GUI with llvmpipe and is in fact fast enough to run gnome.
  • Built-in NVME drive: Block device nodes exist. My system is not deployed on the internal SSD. Actual usuability and performance remain untested.

Devices and Software

  • A linux host where you build the kernel and the bootloader.
  • A M1 Mac Mini. Mine has 10Gbit ethernet.
  • A USB drive to serve as a makeshift root device.
  • A USB-something to USB-C 3.0 cable for sideloading kernel and initrd.
  • A USB keyboard hooked on the Mac Mini.

Compiling

Build the kernel first. Make sure environment variable is set:

  • ARCH=arm64
  • CROSS_COMPILE= the path of the cross compiler tool chain.
  • INSTALL_MOD_PATH= /usr in the thumbdrive root file system.

Verify you have arch/arm64/boot/Image.gz, arch/arm64/boot/dts/apple/t8103-j274.dtb. Run make modules_install and make sure the kernel modules are correctly stored at <thumbdrive>/usr/lib/modules/<version-localversion>.

There’s no good way to generate an initrd without messing around with QEMU userspace emulation or going to another arm64 Linux machine. Since we are already in a chroot, you may configure root passwords, hostname / timezone and other stuff in there as well. See a random Stack Exchange post on how to do this.

Building m1n1 is very straight forward. Modify the first line in Makefile and set ARCH to ${CROSS_COMPILE}, then follow README at https://github.com/AsahiLinux/m1n1. Find compiled m1n1.macho in build/.

Follow AsahiLinux Dev Quick Start to provision disk space from Mac OS, transfer m1n1.macho to 1TR and use kmutil to use m1n1 as a custom boot object, connect Mac Mini to build host with a USB cable.

Sideloading Kernel and Initrd

m1n1 has a bunch of python scripts to interact with a Mac running m1n1. When the Mac mini is connected to the Linux PC it should appear as a USB serial device, something like /dev/ttyACM0.

Set environment variable M1N1DEVICE to /dev/ttyACM0. Invoke proxyclient/tools/linux.py to sideload a kernel like this:

python3 linux.py \
    -b 'root=/dev/sda1 rw earlycon console=tty0' \
    ${kernel_source}/arch/arm64/boot/Image.gz \
    ${kernel_source}/arch/arm64/boot/dts/apple/t8103-j274.dtb \
    /path/to/initramfs-5.16.0-rc1-cth-m1+.img

Bootable m1n1 + Kernel Payload

Use the following command to concatenate m1n1 and other stuff used to boot:

cat m1n1.macho \
    <(echo 'boot-args=root=/dev/sda1 rw earlycon console=tty0') \
    ${kernel_source}/arch/arm64/boot/Image.gz \
    ${kernel_source}/arch/arm64/boot/dts/apple/t8103-j274.dtb \
    /path/to/initramfs-5.16.0-rc1-cth-m1+.img \
> /tmp/m1n1+kernel.macho

The combined boot object can be used to boot directly into Linux without another PC.

Notes

Booting from iSCSI

At boot pass parameters root=UUID=XXXX ip=dhcp rd.iscsi.initiator=iqn.<host-iqn:identifier> netroot=iscsi:<server-ip>::::iqn.<server-iqn:target> instead of a normal root=/dev/XXX. Also make sure your dracut.conf includes appropriate modules:

add_dracutmodules+=" iscsi lvm "
# I am using LVM on iSCSI. Ignore if just using normal GPT
install_items+=" /usr/bin/lsusb /usr/bin/lspci /usr/bin/iperf3 "
# Add additional files as you wish

I have effectively included as many core system utilities as possible in case I need to operate on the root device before leaving initrd.

HDCP everywhere under Mac OS / 1TR

Due to lack of desk space and monitors, my Mac mini is hooked up to an HDMI capture card. If your capture card can’t terminate HDCP encryption, anything the Mac Mini displays when running Mac OS / 1TR will be unviewable over the capture device even when there are no “copyrighted” contents being displayed at all. (Dammit. Apple. Why?)

Once M1N1 loads, the capture card should report non-HDCP encrypted content. After Linux kernel taking over the framebuffer, HDMI capture will remain unencrypted and functional.

A Note on SIT Tunnel at Home

This blog article records the progress of setting up IPv6 tunnel to Hurricane Electric at home where my ISP has no plan in sight to provide prefix-delegated IPv6 access over a PPPoE.

Shorthands and Assumptions in This Note

  • eth0 connects to the Internet via IPv4. This note shall also apply to encapsulated interfaces, e.g. vlan15@eth0, lte0 or pppoe0.
  • eth1 connects to local LAN. Similar as above, the process shall be the same when LAN side is a VLAN or bridge (or both).
  • tun0 denotes the sit tunnel interface created in this step.
  • A line beginning with # denotes comments in the configuration notes.

Setting up the Tunnel Interface on ER-X

If registered correctly on TunnelBroker, it should provide the following information:

  • Logical address at local endpoint, e.g. 2001:444:111:222::2/64
  • Logical address at remote endpoint, e.g. 2001:444:111:222::1/64
  • IPv4 address at remote endpoint where encapsulated traffic is sent, e.g. 66.220.18.42, the HE tunneling endpoint in Paris.
  • A routable prefix for client side delegation, e.g. 2001:444:112:222::/64. This is usually different from the v6 addresses for the endpoint, and HE will show segments of the prefix in bold.

Now fill ER-X configuration nodes with corresponding information and default routing for IPv6:

interfaces:
    tunnel:
        tun0:
            address: [Fill logical v6 address in CIDR at endpoint]
            description: [Give a name to this tun]
            encapsulation: sit
            local-ip: [Fill in IPv4 address at eth0]
            remote-ip: [Fill in IPv4 address at tunneling endpoint]
protocols:
    static:
        interface-route6:
            ::/0:
                next-hop-interface: tun0
            # This creates a default IPv6 routing table entry that
            # routes all non-link-local address to the tunnel.

At this point, one should be able to ping any IPv6 address from the ER-X. If this is working, continue to instruct the LAN interface to delegate the prefix

interface:
    <path-to-interface-config-node>:
        ipv6:
            dup-addr-detect-transmits: 1
            # Stateless SLAAC configuration might produce identical
            # IP addresses. This allow the network to detect whether
            # a stateless address already exists.
            address:
                autoconf
                # Set autoconf to allow stateless delegation by SLAAC
            router-advert:
                prefix:
                    [Fill routable delegated prefix here]:
                        autonomous-flag: true
                        # Instructs computers on this network to auto
                        # discover DNS servers
                        on-link-flag: true
                        # Indicates that this prefix exists on the
                        # same Ethernet link, i.e. these addresses
                        # does not require routing

IPv6 enabled devices shall now receive globally unique IPv6 address assigned via SLAAC and prefix delegation.

Subsequent Steps

  • Confirm IPv6 assignment on LAN devices
$ ip addr
<------ MORE INTERFACES REDACTED ------>
2: eno1:  mtu 1480 qdisc fq_codel state UP group default qlen 1000
     link/ether XX:XX:XX:XX:XX:XX brd ff:ff:ff:ff:ff:ff
     inet 192.168.5.4/24 brd 192.168.5.255 scope global dynamic noprefixroute eno1
        valid_lft 80444sec preferred_lft 80444sec
     inet6 2001:470:d:XXXX:XXXX:XXXX:XXXX:dfd3/64 scope global dynamic noprefixroute 
        valid_lft 2591976sec preferred_lft 86376sec
     inet6 fe80::be40:XXXX:XXXX:XXXX/64 scope link noprefixroute 
        valid_lft forever preferred_lft forever
<------ MORE INTERFACES REDACTED ------>
  • Trace IPv6 connections to an IPv6 enabled website
$ traceroute -6 ac.cth451.me -n
 traceroute to ac.cth451.me (2606:4700:30::681c:1b16), 30 hops max, 80 byte packets
  1  2001:470:d:XXXX:XXXX:XXXX:XXXX:XXXX  0.415 ms  0.533 ms  0.624 ms
  2  2001:470:c:XXXX::1  185.353 ms *  203.802 ms
  3  2001:470:0:9d::1  178.603 ms  167.001 ms  189.255 ms
  4  2001:504:0:3:0:1:3335:1  196.520 ms  179.737 ms  196.117 ms
  5  2400:cb00:12:1024::6ca2:d61d  185.036 ms 2400:cb00:12:1024::6ca2:d614  175.573 ms 2400:cb00:12:1024::6ca2:d608  185.263 ms
  • It is advisable to setup network wide firewall on the router, as addresses can be reached by any other IPv6 connected devices from the Internet.

Further Notes

  • The sit tunnel shall also work if setup correctly on any other router or even a personal computer with public IPv4 address. I am unable to replicate the settings on a Linux router via raw commands as I do not own a linux machine with public IPv4 address.
  • I am not sure if the method would work if the local endpoint is behind NAT. This scenario will be experimented on after I return to campus.

“Certified” Androids?

The story begins with the moment I bought Cytus II on the very first day of its release from google play, expecting some real music gaming on my Surface Pro 3 running homebrew Android 7.1 (yes, that’s totally possible and it runs suprisingly fine with all hardware buttons and touchscreen working). However the game quits immediately and no logs are shown through the adb interface. I contacted Rayark for support and got reply like this:

Cytus II might only be compatible for native Android devices at the moment. Please also check if you have installed Xposed, firewalls, block ads, or any rooting software. If yes, these may effect the performance of the game. We’d like to suggest you to remove these software to ensure the game runs smoothly and properly.

(Probably) 3 weeks after the game’s release, I ended up playing Cytus II on my crammed 5′ mobile phone, not by choice though. Someone asked me, why I can’t buy a regular Android tablet or an iPad to do the same job. Well that could be an option if those people would donate in any means for a new one, and I really don’t think utilizing an existing hardware piece could cause any troubles for commercial devs and companies (Google: really?).

After some searches I reached conclusion that Cytus II has integrated an “compliance check” called SafetyNet and the underlying Compatibility Test Suite (CTS), a framework introduced by Google to verify if any android device falls into the category of “compatible”. Since Surface Pro 3 has never got an official Android release (it’s a Microsoft thing, of course) and the base android x86 is shipped with root and development mode on, there is probably no way that any CTS tests on Surface Pro 3 would pass any time sooner.

Just earlier this day, I saw the news that Google is attmpting to block Gapps from running on “uncertified” devices, where modding the android device or unlocking bootloader would void the “certified” status. There’s even a webpage letting people to “register” their Androids with their device identifier which is absolutely not working after I attempted to register my Surface only getting an unknown error.

I would not blame Rayark for their attempt to place a layer of piracy protection on such a nice game while hurting the ones who modded their devices properly exactly to play these games. My question is: If such a lockdown is so important that this system had been deployed to thousands of android software by now, why make android an open standard? Why not switch to the Apple production mode if a centralized control force seems so vital to the whole android community? Such blockage wouldn’t be easily tolerated if the so called “register uncertified devide page” is just a lie, and I believe there will be a solution to circumvent such unreasonable restrictions eventually. Before that, the vast population of “uncertified” androids and modders wouldn’t be so comfortable and I might really need to ask myself: Why I should spend 3 months porting an open source OS to a new device just to find nothing should works by design.

Sakura island blender render

Sakura Island is the spawn point in my minecraft server. Of course it would not be possible to place sakura trees in game, so what’s actually in Minecraft is pink wool. Nevertheless they still look nice from far away.

I tried water caustics with refraction and fresnel effects, which make water more realistic at the cost of increased samples and render time. Also I attempted a manual material override on tree leaves, However that seems worth it.

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. Minecraft creation by me, exported with mineways on wine, rendered on blender with cuda. As usual, full resolution of 4096×4096 is available on pixiv.

Sakura Island At Night
Rendered in Blender

[Errata] Windows upgrade to 1703 broke grub

What happened?

After windows upgrading itself to release 1703 aka. the creator update, Grub bootloader can no longer start and says error: unknown filesystem and dropped in to grub rescue shell. Windows boots up fine from firmware memu.

TL; DR

Manually doing hexediting just taught me another valuable lesson: @Windows has the worst upgrade handling.

— @cth451 July 28, 2017

  1. Prepare a USB linux live environment.
  2. Find the correct partition number, it might have been changed during upgrade.
  3. Look for grubx64.efi in your EFI system partition and use a hex editor to open it
  4. Search for /boot/grub and notice the (,gptN) where N is your original partition number for linux OS.
  5. Change N to the new parition number, save the file and reboot.

Analysis

Before upgrade, my partition scheme was:

  • sda1 fat32: EFI system parition
  • sda2 ext4: Archlinux root filesystem
  • sda4 ntfs: Windows drive C
  • sda3 ext4: AOSC OS root filesystem

Partition numbers were not exactly in physical order of data, for I shrinked Archlinux to make room for Windows installation. However after upgrade, it became:

  • sda1 fat32: EFI system parition
  • sda2 ext4: Archlinux root filesystem
  • sda3: Windows drive C
  • sda4: Windows recovery environment
  • sda5: AOSC OS root filesystem

So what did windows do down there? It shrinked drive C and made a little recovery partition, then re-sorted the partition numbers. However grub bootloader looks for partitions by partition number. It tried parition 3 (which was a valid linux filesystem), but it’s an ntfs now.

This partition number is hardcoded into the bootloader efi image, so we need to manually hexedit it.

Triangle Garden (Blender Rendering Teaser)

New features I discovered in blender: manual (and sophiscated) water surface reflection controls! I tried some techiques from the computer graphics class I’m taking this semester, and the result is over fancy. I forgot to ask blender to keep rendering specs. The only specs I remembered are 4096 samples per pixels and that I started blender before I went to sleep, and the process actually had finished before I woke up the next day for classes.

Anyways, here’s the pic. As usual, the image here is downscaled so that the page loads in acceptable time. Shoot me an email for the full image at 4320×2160 if you have 4K (?) displays. See Minecraft Server Page to join the server where I built this thing.

Triangle Garden Rev2 2:1 2160x1080
Triangle Garden Rev2 2:1 2160×1080

Minecraft 1.8 Forge API IBlockState 与渲染

在 Minecraft 内部,方块也可以看作是可放置的物品,使用 Item 类的静态方法 getItemFromBlock(Block someBlock) 可以获得对应的物品。向游戏内注册该方块也是类似的使用 GameRegistry.registerBlock(Block blockToReg, String unlocalizedName)。然而在处理渲染时需要多额外的一步。

以下为 Item 渲染

String nameToReg = MOD_ID + ':' + item.getUnlocalizedName().substring(5);
Minecraft.getMinecraft()
        .getRenderItem().getItemModelMesher()
        .register(item, 0,
        new ModelResourceLocation(nameToReg, "inventory");

以下为 Block 渲染

Item item = Item.getItemFromBlock(block);
// 从 Block 得到对应的 Item
String nameToReg = MOD_ID + ':' + item.getUnlocalizedName().substring(5);
Minecraft.getMinecraft()
        .getRenderItem().getItemModelMesher()
        .register(item, 0,
        new ModelResourceLocation(nameToReg, "inventory");

区别主要在于多了一句从 Block 获得 Item。虽然是以 Item 注册了渲染,实际上这里和 Item 处理渲染的方法并不相同。

assets/<MOD_ID>/blockstates/<blockUnlocalizedName>.json

我所写的 mod 中许多方块都会根据不同的 BlockState 改变渲染模型,而 BlockState 中不同 IProperty 组合所对应状态所对应的模型在这里指定。关于 BlockState,见上一篇 Forge API 详解。我在这里以 TransitRailMod 的电缆架作为例子。

在源代码中,WirePanel 是 CustomDirectionalBlock 的子类,因此继承其 EnumFacing FACING 的属性,这个方块又根据电缆架上是否插了日光灯改变 EnumBool LAMP 属性、是否是封闭的改变 EnumBool SHUT 属性。因此罗列一下: Continue reading Minecraft 1.8 Forge API IBlockState 与渲染

Minecraft 1.8 Forge API BlockState 与红石详解

由于网上找到的 Minecraft Forge 文档实在是少的可怜,我决定自己写一份以防未来的自己忘记。具体的实践可以在我的客运铁路 mod 见到。

红石的更新本质上也是方块更新,因此当一个方块想要检测红石信号,处理的代码应当放在 Block 类的 onNeighborBlockChange(World worldIn, BlockPos pos, IBlockState state, Block neighborBlock) 方法。

BlockStateIBlockState

可以在方块的 Class 建立 BlockState 以允许方块拥有不同的状态 (IProperty),不同的状态可以在 assets/<MODID>/blockstates/<block_instance_name>.json 中定义不同的方块模型。

每一个 IProperty 拥有自己的 name 和可以取到的有限个值,这些值在 Java 内部以 enum 的形式实现。关于 enum 变量类型,可以在王八壳的 javadoc 上找到。

一个方块的 BlockState 可以包含一个或多个 IProperty,可以使用已经定义好的 IProperty 子类(PropertyDirection, PropertyBool)也可以自己定义 enum 建立 PropertyEnum

Class: public CustomBlock extends Block

通过以下方式建立 IProperty 及其子类的实例:

  • public static final PropertyDirection FACING = PropertyDirection.create("facing", EnumFacing.Plane.HORIZONTAL);
    建立新的方向属性,名称为 "facing" ,可以取的值为平面方向的朝向,为 EnumFacing 下的 NORTHEASTSOUTHWEST
  • public static final PropertyBool POWERED = PropertyBool.create("powered");
    建立布尔型只能为 truefalse 的方块属性。

Continue reading Minecraft 1.8 Forge API BlockState 与红石详解

Time-based One Time Password Algorithm

Follow the algorithm

As implemented in Google Authenticator app, you’ll need:

  • A shared secret key K, which is base32 encoded
  • a specific time period since UNIX epoch T

I’ll use K = GEZDGNBVGY3TQOJQGEZDGNBVGY3TQOJQ and T = 1450235092 (which is Wed, 16 Dec 2015 11:04:52 CST) for example. In the Google Authenticator App, both K and T are passed to TOTP as hexadecimal values (or byte arrays).

Since K is a base32 encoded string, we’ll decode it to byte format. Password changes every 30 seconds, T should be divided by 30, tuncated to integer and converted to hex and padded to 16 hexadecimal digits.

Continue reading Time-based One Time Password Algorithm