Handling multiple chunks of POST data

[Sriharsha Setty]

Hello,
I want my application to handle requests sent through `curl' like this:

curl -X POST -d @def.json -d @image.jpg http://localhost:8080/json

The first part of the POST chunk is a JSON snippet read from a file and the
other is an image file.

I believe request.body gives me both these chunks as an single entity. Is
there a way I can handle both these separately ?


Thank you,

[m***@gsites.de]

Post by Sriharsha Setty
Hello,

Curl uses url-encoding to transmit data with the -d option. This is not
suited for binary data.

Post by Sriharsha Setty
I believe request.body gives me both these chunks as an single entity. Is
there a way I can handle both these separately ?

Form data is parsed into request.POST (or request.forms and request.files)
but only if the POST body is a valid application/x-www-form-urlencoded or
multipart/form-data stream.

[Sriharsha Setty]

Post by m***@gsites.de
Form data is parsed into request.POST (or request.forms and request.files)
but only if the POST body is a valid application/x-www-form-urlencoded or
multipart/form-data stream.

Is there a way I can address each part of the multipart data in my
application? Depending on what my POST data is, I want process it in a
certain way. Eg: if it is a JSON snippet, do something and if it is a jpeg
do something else.


/harsha

[m***@gsites.de]

Post by Sriharsha Setty

Post by m***@gsites.de
Form data is parsed into request.POST (or request.forms and
request.files)
but only if the POST body is a valid application/x-www-form-urlencoded or
multipart/form-data stream.

Is there a way I can address each part of the multipart data in my
application? Depending on what my POST data is, I want process it in a
certain way. Eg: if it is a JSON snippet, do something and if it is a jpeg
do something else.

This is not differed from normal file upload fields in HTML forms. See
curl docs:

-F/--form <name=content>
(HTTP) This lets curl emulate a filled-in form in which a user
has pressed the submit button. This causes curl to POST data
using the Content-Type multipart/form-data according to RFC2388.

You can pass a content-type, too. For example:

curl -F "data=@data.json;type=application/json" example.com/submit

And here is a (very basic) from processing example:

from bottle import request, post
@post('/submit')
def submit():
data = request.files['data']
ctype = data.headers['content-type']
if ctype == 'application/json':
process_json(data.file.read())
elif ctype = 'image/jpeg':
process_jped(data.file.read())

(untested and insecure, do not copy&paste !!!)

[Sriharsha Setty]

Post by m***@gsites.de
from bottle import request, post
@post('/submit')
data = request.files['data']
ctype = data.headers['content-type']
process_json(data.file.read())
process_jped(data.file.read())
(untested and insecure, do not copy&paste !!!)

Thank you! I got it work using -F.

/harsha

[Fred Eisele]

Post by Sriharsha Setty
Hello,

As Marcel pointed out there are some problems using the --data
(--data-ascii -d) option.
* It strips data (\r \n) from the file indicated with the @.

This stripping of data can be prevented with a different option.
curl -X POST --data-binary @def.json --data-binary @image.jpg
http://localhost:8080/json

* It concatenates the contents of the files separated by an '&'

Post by Sriharsha Setty
The first part of the POST chunk is a JSON snippet read from a file and
the other is an image file.

In the specific example you mention there is no such assurance that the '&'
will only occur between the two file's payloads.

Post by Sriharsha Setty
I believe request.body gives me both these chunks as an single entity. Is
there a way I can handle both these separately ?
You can make use of another variant on the --data option to urlencode the

files.
As urlencoding effectively replaces the '&' with '%26' the '&' inserted can
now be properly used to split the request.body.
You will have to use something like the following (python3) in the server.

from urllib import parse
for load in request.body.split(b'&'):
parse.unquote_to_bytes(load)
--

[Marcel Hellkamp]

You replied to a 6 years old question which was answered already. Probably
by accident?

For the record: The best solution would be to use curl form handling:

-F/--form <name=content>
(HTTP) This lets curl emulate a filled-in form in which a user
has pressed the submit button. This causes curl to POST data
using the Content-Type multipart/form-data according to RFC2388.

multipart/form-data file uploads can be accessed via
``bottle.request.files``.
--

[Fred Eisele]

No, it was not an accident.
I have a specific requirement to use the --data* option and not --form.
Once I figured out an answer that fit the additional constraint I thought I
should post it.

But, too be clear, if at all possible the --form option should be preferred.
--