lundi 6 avril 2015

Why does method 'readdirSync' use so much memory when reading directory with big amount of files?

Here is an example of NodeJS code:



var fs = require('fs');

function toMb (byteVal) {
return (byteVal / 1048576).toFixed(2);
}

console.log('Memory usage before "readdirSync" apply: ', toMb(process.memoryUsage()['heapUsed']) + ' MB');

fs.readdirSync('./parseLogFiles/reports');

console.log('Memory usage after "readdirSync" apply: ', toMb(process.memoryUsage()['heapUsed']) + ' MB');


Directory "reports" contains 300.000 files.


I have got the following results:



Memory usage before "readdirSync" apply: 2.01 MB
Memory usage after "readdirSync" apply: 22.38 MB


Why memory using increased more than 10 time (2.01 vs 22.38)?


For "readdir" i have the same results.


Another example:



var fs = require('fs');

function toMb (byteVal) {
return (byteVal / 1048576).toFixed(2);
}

console.log('Memory usage before "readdirSync" apply: ', toMb(process.memoryUsage()['heapUsed']) + ' MB');

var filesList = fs.readdirSync('./parseLogFiles/reports');

console.log('Memory usage after "readdirSync" apply: ', toMb(process.memoryUsage()['heapUsed']) + ' MB');
console.log('Files list size: ', toMb(Buffer.byteLength(filesList.join(''))) + ' MB');


I have got the following results:



Memory usage before "readdirSync" apply: 2.01 MB
Memory usage after "readdirSync" apply: 22.38 MB
Files list size: 11.13 MB


From where come 9,24Mb (22.38 - 11.13 - 2.01) from?


Aucun commentaire:

Enregistrer un commentaire