grpc message size limits causes keyword failures with large DOMs

[frste]

Describe the bug
In case of a website dom which contains a lot of content, keywords like "Get Page Source" may fail with message: "CLIENT: Received message larger than max (4200443 vs. 4194304)"

To Reproduce
Unfortunately I can't provide a public website for testing this behavior

Expected behavior
The called keywords should not fail because of reduced message length limits

Desktop (please complete the following information):

• OS: Windows 11
• Browser: chromium
• Version 140.0.3485.54

Additional context

• The root cause for this problem seems to be the limits of the gprc channel. When setting appropriate max size for the gprc_channel solves the problem for this specific case
• grpc.insecure_channel(f"{self.host}:{self.port}", options=[ ('grpc.max_message_length', 4300000), ('grpc.max_send_message_length', 4300000), ('grpc.max_receive_message_length', 4300000), ],)

Could you please check this case and either adjust the grpc_channel settings to appropriate values for handling large websites (we also have seen cases of even bigger dom content) or make these values configurable.

Kind regards, Frank

[aaltat]

No need to create example, I know what causes that error. GRPC has max limit which it can transfer with one message. The fix is to move to use stream with GRPC messaging model

@allcontributors please add @frste for bugs

[allcontributors]

@aaltat

I've put up a pull request to add @frste! 🎉

[aaltat]

Can you provide a PR to fix the problem?

[frste]

@aaltat I’m not yet familiar with the concept, but I’ll give it a try

[aaltat]

Wait For Response has a stream https://github.com/MarketSquare/robotframework-browser/blob/main/protobuf/playwright.proto#L542 it can serve as an example how protobuf and Python + NodeJS code should be changed.