WIP: fizz: soc/intel/pmclib: Fix boot issue after power loss when using USB-C PD

[movr4x]

FIZZ boards are unable to boot after power loss if USB-C PD is used to initially power them on when CFR Restore AC power after loss (power_on_after_fail) is set to:

• Power off (S5) - Always.
• Previous state - Only after a clean shutdown.

Both options result in Intel chipset after G3 being set by coreboot to keep the device off when power is restored (Previous state only after a clean shutdown).

The issue is likely related to some sort of miscommunication between chipset and ChromeEC, perhaps due to USB-C PD induced delays (barrel jack not affected), where ChromeEC fails to force the chipset to bring the system up and gives up. Since this affects EC RO there is no safe way of mitigating this issue from EC side.

This was reported by users:
MrChromebox/firmware#595 (comment)

The solution is to always make sure that chipset after G3 is set to auto on, and let EC handle the rest if EC manages after G3.

The usual flow for configuring chipset after G3 in coreboot for Intel is as follows:

Intel common PMC Kconfig (src/soc/intel/common/block/pmc/Kconfig) selects:

config SOC_INTEL_COMMON_BLOCK_PMC
[...]
	select HAVE_POWER_STATE_AFTER_FAILURE
	select HAVE_POWER_STATE_PREVIOUS_AFTER_FAILURE
[...]

It also sets:

config POWER_STATE_DEFAULT_ON_AFTER_FAILURE
	default y

HAVE_POWER_STATE_AFTER_FAILURE causes mainboard Kconfig (src/mainboard/Kconfig) to define MAINBOARD_POWER_FAILURE_STATE, either by a default pathway, or by custom selection:

config HAVE_POWER_STATE_AFTER_FAILURE
	bool

if HAVE_POWER_STATE_AFTER_FAILURE

config HAVE_POWER_STATE_PREVIOUS_AFTER_FAILURE
	bool

config POWER_STATE_DEFAULT_ON_AFTER_FAILURE
	bool
	help
	  Selected by platforms or mainboards that want a "default on"
	  behaviour.

choice
	prompt "System Power State after Failure"
	default POWER_STATE_ON_AFTER_FAILURE \
		if POWER_STATE_DEFAULT_ON_AFTER_FAILURE
	default POWER_STATE_OFF_AFTER_FAILURE
	help
	  Provides a default for the power state the system should
	  go into after G3 (power loss). On many boards this can be
	  overridden by an NVRAM option.

config POWER_STATE_OFF_AFTER_FAILURE
	bool "S5 Soft Off"
	help
	  Choose this option if you want to put system into
	  S5 after reapplying power after failure.

config POWER_STATE_ON_AFTER_FAILURE
	bool "S0 Full On"
	help
	  Choose this option if you want to keep system in
	  S0 after reapplying power after failure.

config POWER_STATE_PREVIOUS_AFTER_FAILURE
	bool "Keep Previous State"
	depends on HAVE_POWER_STATE_PREVIOUS_AFTER_FAILURE
	help
	  Choose this option if you want to keep system in the
	  same power state as before failure after reapplying
	  power.

endchoice

config MAINBOARD_POWER_FAILURE_STATE
	int
	default 2 if POWER_STATE_PREVIOUS_AFTER_FAILURE
	default 1 if POWER_STATE_ON_AFTER_FAILURE
	default 0

endif # HAVE_POWER_STATE_AFTER_FAILURE

MAINBOARD_POWER_FAILURE_STATE is then used as a default for configuring chipset after G3, which in the case of Intel can be then additionally changed via CFR power_on_after_fail (src/soc/intel/common/block/pmc/pmclib.c):

void pmc_set_power_failure_state(const bool target_on)
{
	const unsigned int state = get_uint_option("power_on_after_fail",
					 CONFIG_MAINBOARD_POWER_FAILURE_STATE);
	[...]

AMD appears to follow a similar pattern, except it always directly uses MAINBOARD_POWER_FAILURE_STATE (no CFR) (src/soc/amd/common/block/pm/pmlib.c):

void pm_set_power_failure_state(void)
{
	uint8_t pwr_fail = PWR_PWRSTATE;

	switch (CONFIG_MAINBOARD_POWER_FAILURE_STATE) {
	[...]

Some of the possible solutions:

1) Keep EC after G3 configuration coupled to chipset after G3 configuration and make Intel chipset after G3 value be condition based - this is the starting point for this PR.

We define some sort of helper to indicate that EC handles after G3, for example in mainboard Kconfig (src/mainboard/Kconfig):

config HAVE_EC_POWER_STATE_AFTER_FAILURE
	bool

HAVE_EC_POWER_STATE_AFTER_FAILURE can be then selected by:

• EC Kconfig - like in config EC_GOOGLE_CHROMEEC - global for all boards using specific EC, where EC sits in front of managing after G3.
• EC Kconfig with specific implementation allowing to override default EC after G3 - like in config EC_GOOGLE_CHROMEEC_AFTER_G3_STATE (or other custom after G3 implementation) - local for boards which explicitly enable custom way of altering EC after G3.
• Board Kconfig - local for boards which have their own way of altering EC after G3.

Intel (and optionally AMD) PM code is changed to always set chipset after G3 to auto on if EC handles after G3:

	const unsigned int state = (CONFIG(HAVE_EC_POWER_STATE_AFTER_FAILURE) ?
				    MAINBOARD_POWER_STATE_ON :
				    get_uint_option("power_on_after_fail",
				         CONFIG_MAINBOARD_POWER_FAILURE_STATE));

Since at the moment the issue appears to affect only Intel, only this platform is targeted, as a starting point. Also, HAVE_EC_POWER_STATE_AFTER_FAILURE is only enabled for boards which use After G3 State implementation to keep this fix local. This can be changed of course if you find it necessary.

Pros:

• Simple.
• Chipset after G3 always set to auto on if EC handles after G3, regardless of CFR/Kconfig choice.
• Kconfig prompt and CFR power_on_after_fail can be re-used by custom implementations that allow to override EC after G3 - one CFR/Kconfig for both chipset and EC after G3..

Cons:

• Need to directly touch global Intel/AMD PM code.
• Boards where it is not possible to alter EC after G3 still expose Kconfig prompt System Power State after Failure, and optionally CFR, which could be problematic, as changing it will have no effect (EC will override this) and could even cause trouble like the one this PR attempts to fix - the latter could be mitigated by making HAVE_EC_POWER_STATE_AFTER_FAILURE global to all boards where EC manages after G3 (like ChromeEC).

2. Complete decoupling of EC after G3 configuration from chipset after G3 configuration.

We define stuff that is used to configure EC after G3 (including separate prompt), for example in mainboard Kconfig (src/mainboard/Kconfig):

config HAVE_EC_POWER_STATE_AFTER_FAILURE
	bool

if HAVE_EC_POWER_STATE_AFTER_FAILURE

config HAVE_EC_POWER_STATE_OFF_AFTER_FAILURE
	bool

config HAVE_EC_POWER_STATE_ON_AFTER_FAILURE
	bool

config HAVE_EC_POWER_STATE_PREVIOUS_AFTER_FAILURE
	bool

config EC_POWER_STATE_DEFAULT_OFF_AFTER_FAILURE
	bool
	depends on HAVE_EC_POWER_STATE_OFF_AFTER_FAILURE
	help
	  Selected by EC or mainboards that want a "default off"
	  behaviour.

config EC_POWER_STATE_DEFAULT_ON_AFTER_FAILURE
	bool
	depends on HAVE_EC_POWER_STATE_ON_AFTER_FAILURE
	help
	  Selected by EC or mainboards that want a "default on"
	  behaviour.

config EC_POWER_STATE_DEFAULT_PREVIOUS_AFTER_FAILURE
	bool
	depends on HAVE_EC_POWER_STATE_PREVIOUS_AFTER_FAILURE
	help
	  Selected by EC or mainboards that want a "default
	  previous state" behaviour.

choice
	prompt "EC System Power State after Failure"
	default EC_POWER_STATE_OFF_AFTER_FAILURE \
		if EC_POWER_STATE_DEFAULT_OFF_AFTER_FAILURE
	default EC_POWER_STATE_ON_AFTER_FAILURE \
		if EC_POWER_STATE_DEFAULT_ON_AFTER_FAILURE
	default EC_POWER_STATE_PREVIOUS_AFTER_FAILURE \
		if EC_POWER_STATE_DEFAULT_PREVIOUS_AFTER_FAILURE
	default EC_POWER_STATE_DEFAULT_AFTER_FAILURE
	help
	  Provides a default for the power state the system should
	  go into after G3 (power loss). On many boards this can be
	  overridden by an NVRAM option.

config EC_POWER_STATE_DEFAULT_AFTER_FAILURE
	bool "Default"
	help
	  Choose this option if you want to let EC decide
	  power state to put system into after reapplying
	  power after failure.

config EC_POWER_STATE_OFF_AFTER_FAILURE
	bool "S5 Soft Off"
	depends on HAVE_EC_POWER_STATE_OFF_AFTER_FAILURE
	help
	  Choose this option if you want to put system into
	  S5 after reapplying power after failure.

config EC_POWER_STATE_ON_AFTER_FAILURE
	bool "S0 Full On"
	depends on HAVE_EC_POWER_STATE_ON_AFTER_FAILURE
	help
	  Choose this option if you want to keep system in
	  S0 after reapplying power after failure.

config EC_POWER_STATE_PREVIOUS_AFTER_FAILURE
	bool "Keep Previous State"
	depends on HAVE_EC_POWER_STATE_PREVIOUS_AFTER_FAILURE
	help
	  Choose this option if you want to keep system in the
	  same power state as before failure after reapplying
	  power.

endchoice

config MAINBOARD_EC_POWER_FAILURE_STATE
	int
	default 2 if EC_POWER_STATE_PREVIOUS_AFTER_FAILURE
	default 1 if EC_POWER_STATE_ON_AFTER_FAILURE
	default 0 if EC_POWER_STATE_OFF_AFTER_FAILURE
	default 3 # EC_POWER_STATE_DEFAULT_AFTER_FAILURE

endif # HAVE_EC_POWER_STATE_AFTER_FAILURE

In the same file we make the System Power State after Failure prompt be only shown if there is no HAVE_EC_POWER_STATE_AFTER_FAILURE:

choice
	prompt "System Power State after Failure"
	depends on !HAVE_EC_POWER_STATE_AFTER_FAILURE
	[...]

In the same file we also make sure MAINBOARD_POWER_FAILURE_STATE is always set to auto on if HAVE_EC_POWER_STATE_AFTER_FAILURE is set:

config MAINBOARD_POWER_FAILURE_STATE
	int
	default 1 if HAVE_EC_POWER_STATE_AFTER_FAILURE
	default 2 if POWER_STATE_PREVIOUS_AFTER_FAILURE
	default 1 if POWER_STATE_ON_AFTER_FAILURE
	default 0

In something like src/ec/pmlib.h (equivalent for pmclib/pmlib in Intel/AMD) we define enum:

/*
 * Which state do we want to goto after g3 (power restored)?
 * 0 == S5 Soft Off
 * 1 == S0 Full On
 * 2 == Keep Previous State
 * 3 == Default
 * Keep in sync with `config MAINBOARD_EC_POWER_FAILURE_STATE`.
 */
enum {
	MAINBOARD_EC_POWER_STATE_OFF,
	MAINBOARD_EC_POWER_STATE_ON,
	MAINBOARD_EC_POWER_STATE_PREVIOUS,
	MAINBOARD_EC_POWER_STATE_DEFAULT
};

In something like src/ec/cfr.h we define EC specific after G3 CFR:

#if CONFIG(HAVE_EC_POWER_STATE_AFTER_FAILURE)

/* Power state after power loss */
static const struct sm_object ec_power_on_after_fail = SM_DECLARE_ENUM({
	.opt_name	= "ec_power_on_after_fail",
	.ui_name	= "EC Restore AC power after loss",
	.ui_helptext	= "Specify what to do when power is re-applied after a power loss.",
	.default_value	= CONFIG_MAINBOARD_EC_POWER_FAILURE_STATE,
	.values		= (const struct sm_enum_value[]) {
				{ "Default",	    MAINBOARD_EC_POWER_STATE_DEFAULT	},
#if CONFIG(HAVE_EC_POWER_STATE_OFF_AFTER_FAILURE)
				{ "Power off (S5)", MAINBOARD_EC_POWER_STATE_OFF	},
#endif
#if CONFIG(HAVE_EC_POWER_STATE_ON_AFTER_FAILURE)
				{ "Power on  (S0)", MAINBOARD_EC_POWER_STATE_ON		},
#endif
#if CONFIG(HAVE_EC_POWER_STATE_PREVIOUS_AFTER_FAILURE)
				{ "Previous state", MAINBOARD_EC_POWER_STATE_PREVIOUS	},
#endif
				SM_ENUM_VALUE_END					},
});

#endif /* HAVE_EC_POWER_STATE_AFTER_FAILURE */

In EC Kconfig, if EC controls after G3, like ChromeEC, we select:

	select HAVE_EC_POWER_STATE_AFTER_FAILURE

In custom implementations, where we have the ability to change EC after G3, we select values that are supported, and optionally the default value (ChromeEC, by default, officially uses previous state):

	select HAVE_EC_POWER_STATE_OFF_AFTER_FAILURE
	select HAVE_EC_POWER_STATE_ON_AFTER_FAILURE
	select HAVE_EC_POWER_STATE_PREVIOUS_AFTER_FAILURE
	select EC_POWER_STATE_DEFAULT_PREVIOUS_AFTER_FAILURE

Then for boards which use EC to control after G3 (like ChromeEC) we replace CFR power_on_after_fail with ec_power_on_after_fail.

Pros:

• Hidden chipset after G3 Kconfig prompt if EC has after G3 handling, since the former has no effect or can even cause trouble.
• Chipset after G3 Kconfig default value always set to auto on if EC handles after G3.
• Exposed new Kconfig prompt for EC after G3, and optionally new CFR, only has options which are supported - only default if no way of changing EC after G3 or default + possible after G3 options.
• No need to alter global Intel/AMD PM code.

Cons:

• More complex.
• Not altering global Intel/AMD PM code could require users to clear NVRAM to get rid of old power_on_after_fail reference, which can still be altering chipset after G3. (this could be mitigated by forcing global Intel/AMD PM code to always use auto on like in the first solution).
• Will also require updating custom EC after G3 handling, like After G3 State implementation for FIZZ, to support new enum with additional default value.
• Two separate CFR/Kconfig combos in the code for handling after G3.

NOTE

In both cases we still end up with PM code emitting printk(BIOS_INFO, "Set power on after power failure.\n"), which can be confusing to some users, but is technically valid, as chipset after G3 is set to auto on.

[movr4x]

@MrChromebox

To proceed with this further and mark this as ready I will need your input on this, so when you find some spare time please take a look at this.

[MrChromebox]

FIZZ boards are unable to boot after power loss if USB-C PD is used to initially power them on when CFR Restore AC power after loss (power_on_after_fail) is set to:

• Power off (S5) - Always.
• Previous state - Only after a clean shutdown.

Both options result in Intel chipset after G3 being set by coreboot to keep the device off when power is restored (Previous state only after a clean shutdown).

The issue is likely related to some sort of miscommunication between chipset and ChromeEC, perhaps due to USB-C PD induced delays (barrel jack not affected), where ChromeEC fails to force the chipset to bring the system up and gives up. Since this affects EC RO there is no safe way of mitigating this issue from EC side.

so this doesn't make sense to me. If the problem were on the EC side, then the SoC register wouldn't matter, since we wouldn't get to S0. I think we need to figure out why the SoC is deciding not to power up the unit, rather than working around it

[movr4x]

@MrChromebox

Well, I have tried to replicate this using my own USB-C PD power supply, which I have ordered recently, and it works just fine for me with Intel chipset set to keep the device off.

Confirmed with cbmem:

[INFO ]  Set power off after power failure.

I went for a reputable brand, Chicony. Model A22-100P2A - 20V/5A/100W.

My SION always boots fine when I press the power button after forced G3. It does take about 5-6 seconds for the device to bring itself up the first time though.

I am beginning to wonder if the issue is related to some sort of botched PD implementation on the power supply side for cases where it fails (still points to EC as the problem).

so this doesn't make sense to me. If the problem were on the EC side, then the SoC register wouldn't matter, since we wouldn't get to S0. I think we need to figure out why the SoC is deciding not to power up the unit, rather than working around it

The #1 reason why EC is the suspect here is because barrel jack works fine. By fine I mean that it renders SoC after G3 register configuration useless, which is what you would expect to happen when EC is designed to force chipset to do what EC wants, which is the case for ChromeEC.

First let's start with how this Intel register is designed. For Skylake/Kabylake (newer gens might use different location, but the rest should be similar):

GEN_PMCON_B - Offset A4h
CFG Register (Size: 32 bits)
Device 31 - Function 2

Bit 0: AG3E - AFTERG3_EN
Default value: 0h
Access: RW

Description:
Determines what state to go to when power is reapplied after a power failure (G3 state).
0 = System will return to an S0 state (boot) after power is re-applied.
1 = System will return to the S5 state (except if it was in S4, in which case
    it will return to S4-like-state). In the S5 state, the only enabled wake-up
    event is the Power Button or any enabled wake event that was preserved
    through the power failure.
This bit is in the RTC well and is only cleared by RTCRST# assertion.

So coreboot can either:

• Write 0 to signal to the chipset that it should bring the system to S0 when power is restored.
• Write 1 to signal to the chipset that it should keep the system in S5 (with S4 exception) when power is restored, but allow wake events like power button to bring the system up to S0.

And something like this is always being setup by coreboot in Intel/AMD SoC code.

If you set CFR to Power on (S0), then coreboot configures this register to 0, but the device still can remain off because ChromeEC via GPIOs/power rails will keep PCH suspended/powered down if EC internal after G3 logic deems that the device should remain off (OFF reset flag present in EC bbram).

In ChromeEC you can find stuff like:

power/intel_x86.c:

enum power_state power_chipset_init(void)
{
	/*
	 * If we're switching between images without rebooting, see if the x86
	 * is already powered on; if so, leave it there instead of cycling
	 * through G3.
	 */
	if (system_jumped_to_this_image()) {
		if ((power_get_signals() & IN_ALL_S0) == IN_ALL_S0) {
			/* Disable idle task deep sleep when in S0. */
			disable_sleep(SLEEP_MASK_AP_RUN);
			CPRINTS("already in S0");
			return POWER_S0;
		}

		/* Force all signals to their G3 states */
		chipset_force_g3();
	}

	return POWER_G3;
}

power/skylake.c:

__attribute__((weak)) void chipset_set_pmic_slp_sus_l(int level)
{
	gpio_set_level(GPIO_PMIC_SLP_SUS_L, level);
}

enum power_state chipset_force_g3(void)
{
	CPRINTS("Forcing fake G3.");

	chipset_set_pmic_slp_sus_l(0);

	return POWER_G3;
}

board/fizz/gpio.inc:

[...]
GPIO(PMIC_SLP_SUS_L,		PIN(8, 5), GPIO_OUT_LOW)			/* SLP_SUS# to PMIC */
[...]

board/fizz/usb_pd_policy.c:

int pd_check_vconn_swap(int port)
{
	/* in G3, do not allow vconn swap since pp5000_A rail is off */
	return gpio_get_level(GPIO_PMIC_SLP_SUS_L);
}

This works as expected for both barrel jack and USB-C, no auto power up at all, regardless of chipset after G3 configuration. USB-C also requires PD negotiation to power the chipset/CPU, so without a green flag from the EC side, which does the heavy lifting here, the chipset/CPU cannot start on it's own.

If you set CFR to Power off (S5), then coreboot configures this register to 1, but the device can still auto power up because EC can simulate power button press to wake the system (valid wake event) if EC internal after G3 logic deems that the device should auto power on (no OFF reset flag in EC bbram).

File common/power_button_x86.c contains power button state machine and logic.

When image is executed it initializes:

static enum power_button_state pwrbtn_state = PWRBTN_STATE_IDLE;

During initialization stage a hook ensures that initial power button state can be updated:

static void powerbtn_x86_init(void)
{
	set_initial_pwrbtn_state();
}
DECLARE_HOOK(HOOK_INIT, powerbtn_x86_init, HOOK_PRIO_DEFAULT);

/**
 * Set initial power button state.
 */
static void set_initial_pwrbtn_state(void)
{
	uint32_t reset_flags = system_get_reset_flags();

	if (system_jumped_to_this_image() &&
	    chipset_in_state(CHIPSET_STATE_ON)) {
		/*
		 * Jumped to this image while the chipset was already on, so
		 * simply reflect the actual power button state unless power
		 * button pulse is disabled. If power button SMI pulse is
		 * enabled, then it should be honored, else setting power
		 * button to PCH could lead to x86 platform shutting down. If
		 * power button is still held by the time control reaches
		 * state_machine(), it would take the appropriate action there.
		 */
		if (power_button_is_pressed() && power_button_pulse_enabled) {
			CPRINTS("PB init-jumped-held");
			set_pwrbtn_to_pch(0, 0);
		} else {
			CPRINTS("PB init-jumped");
		}
		return;
	} else if ((reset_flags & RESET_FLAG_AP_OFF) ||
		   (keyboard_scan_get_boot_keys() == BOOT_KEY_DOWN_ARROW)) {
		/*
		 * Reset triggered by keyboard-controlled reset, and down-arrow
		 * was held down.  Or reset flags request AP off.
		 *
		 * Leave the main processor off.  This is a fail-safe
		 * combination for debugging failures booting the main
		 * processor.
		 *
		 * Don't let the PCH see that the power button was pressed.
		 * Otherwise, it might power on.
		 */
		CPRINTS("PB init-off");
		power_button_pch_release();
		return;
	}

#ifdef CONFIG_BRINGUP
	pwrbtn_state = PWRBTN_STATE_IDLE;
#else
	pwrbtn_state = PWRBTN_STATE_INIT_ON;
#endif
	CPRINTS("PB %s",
		pwrbtn_state == PWRBTN_STATE_INIT_ON ? "init-on" : "idle");
}

What happens here is:

• If none of the conditions match, then pwrbtn_state is set to PWRBTN_STATE_INIT_ON (no CONFIG_BRINGUP).
• If EC_RESET_FLAG_AP_OFF/RESET_FLAG_AP_OFF condition matches, then pwrbtn_state remains in idle state. Additionally call to power_button_pch_release() is made to ensure no power button press to PCH.
• If EC_RESET_FLAG_AP_IDLE condition matches (only in newer boards, not present in the code above), then pwrbtn_state also remains in idle state.

Then state machine, if pwrbtn_state == PWRBTN_STATE_INIT_ON, will try to initiate power up sequence, which somewhere along the way includes emitting power button press to PCH.

Functions related to emitting power button press to PCH:

static void set_pwrbtn_to_pch(int high, int init)
{
	/*
	 * If the battery is discharging and low enough we'd shut down the
	 * system, don't press the power button. Also, don't press the
	 * power button if the battery is charging but the battery level
	 * is too low.
	 */
#ifdef CONFIG_CHARGER
	if (chipset_in_state(CHIPSET_STATE_ANY_OFF) && !high &&
	   (charge_want_shutdown() || charge_prevent_power_on(!init))) {
		CPRINTS("PB PCH pwrbtn ignored due to battery level");
		high = 1;
	}
#endif
	CPRINTS("PB PCH pwrbtn=%s", high ? "HIGH" : "LOW");
	gpio_set_level(GPIO_PCH_PWRBTN_L, high);
}

void power_button_pch_press(void)
{
	CPRINTS("PB PCH force press");

	/* Assert power button signal to PCH */
	if (!power_button_is_pressed())
		set_pwrbtn_to_pch(0, 0);
}

void power_button_pch_release(void)
{
	CPRINTS("PB PCH force release");

	/* Deassert power button signal to PCH */
	set_pwrbtn_to_pch(1, 0);

	/*
	 * If power button is actually pressed, eat the next release so we
	 * don't send an extra release.
	 */
	if (power_button_is_pressed())
		pwrbtn_state = PWRBTN_STATE_EAT_RELEASE;
	else
		pwrbtn_state = PWRBTN_STATE_IDLE;
}

void power_button_pch_pulse(void)
{
	CPRINTS("PB PCH pulse");

	chipset_exit_hard_off();
	set_pwrbtn_to_pch(0, 0);
	pwrbtn_state = PWRBTN_STATE_LID_OPEN;
	tnext_state = get_time().val + PWRBTN_INITIAL_US;
	task_wake(TASK_ID_POWERBTN);
}

This allows ChromeEC to force the device to power up even with chipset register being set to keep the device off - power button acts as a valid wake event.

When you reconnect power to the device, and it auto powers on, you can see from cbmem for Intel PM section:

[DEBUG]  pm1_sts: 8100 pm1_en: 0000 pm1_cnt: 00001c00
[DEBUG]  gpe0_sts[0]: 00000000 gpe0_en[0]: 00000000
[DEBUG]  gpe0_sts[1]: 00000000 gpe0_en[1]: 00000000
[DEBUG]  gpe0_sts[2]: 00000001 gpe0_en[2]: 00000000
[DEBUG]  gpe0_sts[3]: 00000000 gpe0_en[3]: 00000000
[DEBUG]  TCO_STS:   0000 0000
[DEBUG]  GEN_PMCON: e0040200 0000503b
[DEBUG]  GBLRST_CAUSE: 00000000 00000000
[DEBUG]  PM1_STS: WAK PWRBTN 
[DEBUG]  prev_sleep_state 5 (S5)

Specifically:

[DEBUG]  PM1_STS: WAK PWRBTN 

Informs that the device was powered on because of power button wake event, even if you did not press it (EC did)

The way I see it:

1. Power is restored after power loss via USB-C PD, powering up EC/CR50. This is also where EC can automatically try to boot the device due to it's own internal after G3 logic (skip step 2 if it does).
2. EC waits for wake event like power button press.
3. EC does USB-C PD negotiation and enables appropriate GPIOs/power rails to power the chipset and the main CPU.
4. EC likely fails to communicate to the chipset a valid wake event, which can bypass chipset after G3 option. The chipset ends up keeping the main CPU in S5, as instructed by it's own after G3 option.
5. EC sees no reaction and puts the device in OFF state again (LED goes off, etc.).
6. Go to step 2.

So when this happens the system very likely does not reach S0 at all and is stuck between "fake G3" and S5.

While it is possible that the chipset is doing something weird here, I would say everything points to EC fault, since EC has all the tools it needs to force the chipset to power the device to S0.

Are you able to replicate this issue on your TEEMO, like the other two users?