Power Bi Unexpected EOF

[jacobherner]

Preflight Checklist

• I have installed the latest version of Power Query SDK.
• I have checked existing resources, including the common issues and the release notes.
• I have searched for similar issues.

Power Query SDK

0.4.0

Regression From

No response

Platform

Windows

Architecture

x64

OS Version

No response

VSCode version

1.89.1

PQSdkTool Path

No response

Bug Description

In my custom connector I have implemented a function which returns a large amount of data by using the list.generate function in order to make a series of web requests to get paginated data from a web service. The list of tables of records is then combined into a single table. The issue is that I sometimes encounter the error in the screenshot below when loading the data. However, the web service that furnishes the data does not report any errors. It seems as though power bi is getting incomplete data somehow or is closing it's connection for some reason. From the perspective of the web service, it gets a series of requests for some of the pages of data and then simply stops receiving requests after the error is encountered in power bi. I will also add that the data is not changing between attempts that succeed and those that fail.

The code in question is here:

Steps to Reproduce

Open the connector in power bi desktop.
Load a table that utilizes the function to get paginated data.
Wait for data to load or fail (the failure is not always consistent).

Actual Experience

Expected Experience

Sometimes all the data loads from all the pages and everything works as expected.

Additional Context

These "Microsoft.Mashup.Container" logs which show the issue from the screenshot.
PowerBI with EOF error.zip

[ptyx507x]

This could be due to the logic behind your M script or how the data source behaves with the logic that you have put in place, but it doesn't seem to be related to the SDK itself.
One thing that you can try to do is to tests your scenarios inside of Power BI Desktop. That is, having your full logic inside of Power BI Desktop, without the use of a custom connector, and see how it behaves inside of Power BI Desktop with just pure M script. The SDK and Power BI Desktop use the same engine behind the scenes, so assuming that you don't need any special extension-based functions you can just replicate all your logic in Power BI Desktop for debugging and troubleshooting purposes. If you are getting the same behavior in Power BI Desktop, then try to check what responses you're getting on your pagesList first. If you are able to create a repro that we can run on our side using the TripPin data source from our tutorials, that would allow to dig further into this if it is indeed an issue with the Power Query engine.

Something else to take into consideration is that your logic has a Table.RowCount which forces a full scan of your table. This effectively breaks lazy evaluation for your pagination, so you'll always be reading the whole table every time which might be quite an expensive thing to do for big queries that have thousands of pages.

Here's the documentation that we have for Pagination:
https://learn.microsoft.com/en-us/power-query/helper-functions#tablegeneratebypage

This function tries to leverage lazy evaluation so it only tries to preview or read the rows needed for the data preview in your Power Query editor and your whole get data experience.

Hope this helps and we look forward to hearing from you.