Tesla does not properly encode URLs

[feld]

I believe this is a bug with Tesla as I would expect the HTTP client/wrapper to handle this automatically.

Take a simple URL with a space in it as an example: http://127.0.0.1:8080/foo/file 1.txt

Let's capture the request with netcat or whatever you prefer:

Tesla.get("http://127.0.0.1:8080/foo/file 1.txt")

> nc -l 127.0.0.1 8080
GET /foo/file 1.txt HTTP/1.1
content-length: 0
host: 127.0.0.1:8080

The space is still there!

Now let's try with curl

> curl "http://127.0.0.1:8080/foo/file 1.txt"
curl: (3) URL rejected: Malformed input to a URL function

Aha, it refused the request. Let's fix it.

curl "http://127.0.0.1:8080/foo/file%201.txt"

GET /foo/file%201.txt HTTP/1.1
Host: 127.0.0.1:8080
User-Agent: curl/8.7.1
Accept: */*

The path is properly encoded; curl wouldn't even attempt the invalid request.

I have tried with Hackney, Gun, and Finch. The result is the same.

Is this an oversight, or does Tesla expect to be provided URLs that are already properly encoded?

[feld]

Simplest workaround is to write a Middleware that takes the URL and runs it through URI.decode and then URI.encode again to be certain it's encoded correctly.]

edit: actually you need to handle path and query params separately -- they need URI.decode_query and URI.encode_query or query params with spaces will be %20 instead of +, etc

[yordis]

@feld my personal stand is that, if there something that belongs to the HTTP specification, but a given Adapter isn't following; then I am open to "correct" the situation in Tesla.

Ideally, Tesla isn't stepping into Adapter territory as much, it is just the last step in the middleware.

I see Tesla like Plug, just about the datastructure and architecture to allow everything else todo the work, so it is more about the middleware and tesla.env than anything else.

Being said, please share your perspective, and expectations.

Simplest workaround is to write a Middleware that takes the URL and runs it through URI.decode and then URI.encode again to be certain it's encoded correctly.

That is the tricky bit, that middleware must be the last one because somebody could screw up with the URL in another middleware after/before ...
So should that be a Adapter concern? Or should we have a function before the adapter that normalize the situation?

I am open to ideas, as long as we stick to proper HTTP spec.

[feld]

I think at a minimum it just needs to be very clearly documented that proper URL-encoding is not handled at all by Tesla.

That is the tricky bit, that middleware must be the last one because somebody could screw up with the URL in another middleware after/before ...

I don't know why you'd be altering the URL in other Middlewares but I suppose there are scenarios where people have decided a Middleware is the right solution?

[yordis]

I don't know why you'd be altering the URL in other Middlewares but I suppose there are scenarios where people have decided a Middleware is the right solution?

Personally, I haven't come across, but, you know, Murphy's Law

[yordis]

@feld any thoughts on the solution here?