I’m currently working on a little “Travel Diary” side project to learn more about NodeJS, Javascript frameworks and Fullstack development in general. I’ve started off by using the traditional MEAN stack (MongoDB, Express, Angular and Node), but will be switching out Angular in the future to explore other frontend frameworks.
One of the requirements of this project is to include some sort of file handling between the client and the server. So since this is a travel diary, I decided that for each destination the user should be able to upload an image that will be displayed on the destination details page.
Implementing this functionality requires the following technical steps:
- Allow the user to upload an image
- Send the image to the server
- Receive the image and save it to a database
- Get the image from the server and display it
Upload an Image
So I start off in the Angular frontend to see how to allow the user to upload an image. I opted to use the ng-file-upload plugin to accomplish this because it has a host of useful features (such as drag and drop, progress indicator, resumable uploads etc.) and it was the first search result in Google – which is usually a good thing. The plugin comes with a custom directive which allows the user to select and upload the file[s], and a custom service which handles sending the file to the server. So let’s take a look at some of the code:
First of all I need to inject the ‘ngFileUpload’ module into my travel diary’s destination details module:
angular.module('travelDiary.destinationDetails', ['ngFileUpload']);
Then I can use the custom directive in the HTML like so:
<button ngf-select="destDetCtrl.setDestinationImage($file)" ngf-pattern="'image/*'" class="btn"> Upload image </button>
The ngf-pattern attribute specifies which content-types (aka MIME types) should be allowed in the file upload, and the ngf-select attribute specifies which controller function to run after the file has been uploaded (and passes the file to the function using $file). The ng-file-upload plugin provides lots of other options for the directive but I just want to keep it simple at this point and get it working.
Send the Image to the Server
Now that the user can upload an image, I need to actually send the image to the server. In the DestinationDetailsController controller I add the setDestinationImage function:
function setDestinationImage(file) { DestinationDetailsService.setDestinationImage(vm.destination._id, vm.destination.name, vm.destination.description, file); }
The controller should only contain code which maps the view to our model, the actual business logic (in this case, sending the file to the backend) should be contained in a service. So the setDesinationImage function in our controller, just acts as a wrapper for the service’s function and also sends along some extra data about our destination apart from the file.
In the DestinationDetailsService service I need to inject ng-file-upload’s ‘Upload‘ service a so that I can use it in setDestinationImage().
angular.module('travelDiary.destinationDetails').service('DestinationDetailsService', DestinationDetailsService); DestinationDetailsService.$inject = ['$http', 'Upload']; function DestinationDetailsService ($http, Upload) { var DestinationDetailsService = { setDestinationImage : setDestinationImage }; function setDestinationImage(id, name, description, file) { Upload.upload({ method: 'POST', url: `/api/destinations/${id}`, data: { name: name, description: description, file: file } }).then(function (resp) { console.log('Success ' + resp.config.data.file.name + 'uploaded. Response: ' + resp.data); }, function (resp) { console.log('Error status: ' + resp.status); }, function (evt) { var progressPercentage = parseInt(100.0 * evt.loaded / evt.total); console.log('progress: ' + progressPercentage + '% ' + evt.config.data.file.name); }); } return DestinationDetailsService; }
Upload.upload() handles sending the files to the server using the multipart/form-data MIME type. I won’t dive into the different MIME types for form-data in this post; at this point it’s enough to know that any time a form includes some sort of file input, the MIME type needs to be set to multipart/form-data.
In order to configure the server request, Upload.upload() takes an object as a parameter which defines the required options. First I specify the HTTP method to be used and I want it to be a POST request since I am sending data. Next I specify the URL to which I want to send the data. Using ES6’s new template literals, it’s super easy to add a dynamic id to the URL instead of having to use silly string concatenation. Finally I specify the data that will be sent where each value represents a field or a file. The function returns a promise so I just specify what should happen once response is successful, erroneous or sends an update.
At this point, the frontend part for uploading and sending the image is done. Next I need to create the endpoint in our NodeJS backend which will handle the request.
Process the Image
Receiving the request, extracting the image data and saving it to the MongoDB database is all handled in the NodeJS/Express server. In order to make this task a bit easier, I used the Multiparty library to facilitate processing the multipart form that is being sent. I included Multiparty in my NodeJS DestinationsController using require:
const multiparty = require('multiparty');
And here’s the entire code for using multiparty to handle the image processing:
// POST - Update destination details app.post('/api/destinations/:destination_id', function(req, res) { var destination = { _id: req.params.destination_id } var form = new multiparty.Form(); form.on('error', function(err) { console.log('Error parsing form: ' + err.stack); }); // Parts are emitted when parsing the form // Parts are Readable Streams form.on('part', function(part) { if (!part.filename) { if (part.name === 'name') { destination.name = ''; part.on('data', function (chunk) { destination.name += chunk.toString(); }); part.on('end', function () { part.resume(); }); } else if (part.name === 'description') { destination.description = ''; part.on('data', function (chunk) { destination.description += chunk.toString(); }); part.on('end', function () { part.resume(); }); } part.resume(); } if (part.filename) { destination.image = { contentType: part.headers['content-type'] } let bufs = []; part.on('data', function (chunk) { bufs.push(chunk); }); part.on('end', function () { destination.image.data = Buffer.concat(bufs); part.resume(); }); } }); form.on('close', function() { Destination.update( { _id : req.params.destination_id }, destination, function(err, destination) { if (err) res.send(err); }); res.end('File uploaded'); }); // Parse req form.parse(req); });
Let me explain what’s going on here. Whenever a POST request is received on the specified URL I create a new Multiparty form object, add eventHandlers on the form to be able to process ‘parts’ (which the form emits whilst receiving data) and finally tell the form to process the request using form.parse(req). The key thing to note is that the ‘parts’ that are emitted by the ‘form’ are actually streams so they require their own set of eventHandlers to process the data which is being streamed through. I completely missed this little detail whilst reading the documentation for Multiparty and got stuck for hours. After solving the Sherlock Holmes’ level mystery, the rest of the steps became pretty clear:
- Check if the received part is a file or a normal field (like name or description) by checking whether it has a filename property.
- If it does not have a filename property, then it is a normal field.
- Identify which field from the name property.
- Process the stream, and assign the first chunk directly to the required variable.
- If it has a filename property, then it is a file.
- Get the content type using part.header[‘content-type’] (if this was passed along with the file).
- Create an empty array.
- Process the stream, but this time each new chunk is pushed to the newly created array.
- When the stream is finished processing, use Buffer.concat() on the array to combine each chunk into a single Buffer object.
- When the form is finished processing, save the destination object (including the image stored as a Buffer) to the database.
Display the Image
Now that the image is saved in the database all that’s left is to display the image whenever the user visits the destination page. I could just send the complete destination object to the client when a destination is requested, but there’s just one small issue. In order to display an image it needs to be encoded in a format that the browser understands. One such encoding is base64 – however – our image is currently stored in the database as a Buffer.
So I can either send the destination object as is (i.e. with the image as a Buffer) and convert to base64 on the AngularJS frontend or do it in the NodeJS backend. I decide to convert the image Buffer in the Node server before sending it to the client as it only takes a single line of code and saves me from having to do it in every application that uses this API.
// GET - Get a specific destination by id app.get('/api/destinations/:destination_id', function(req, res) { // lean() used to return Javascript object in callback rather than model object Destination.findById(req.params.destination_id).lean().exec(function(err, destination) { if (err) res.send(err) destination.image.data = destination.image.data.toString('base64'); res.json(destination); }); });
This keeps the frontend code clean and simple. All that I need to do in the Angular app is add a function to the DestinationDetailsService which makes a GET request to the server:
function getDestinationDetails(id) { return $http.get(`/api/destinations/${id}`); }
Call the getDestinationDetailsFunction from the DestinationDetailsController to map the response to the destination object:
function getDestinationDetails(id) { DestinationDetailsService.getDestinationDetails(id) .success(function getDestinationDetailsSuccess(data) { vm.destination = data; }) .error(function getDestinationDetailsError(data) { console.log('Error: ' + data); vm.destination = null; }); }
And finally, add an <img> tag to the DestinationDetailDirective’s template, and use ng-src (regular src would not allow me to interpolate values from the controller) to set the content type and the base64 encoded image string to the values in the model.
<img ng-src="data:{{destDetCtrl.destination.image.contentType}};base64,{{destDetCtrl.destination.image.data}}">
If you want to see the full source code – check out my Travel Diary project over on GitHub! It’s an ongoing work-in-progress where I explore development using the MEAN stack and several other frontend technologies.
Leave a Reply