Mongoose many-to-many complement set

I'm attempting to use Mongoose to build a many-to-many relationship between users.

There are two types of users: teachers and students. A teacher can be in charge of an arbitrary number of students. On the other hand, a student can corresponds to multiple teachers.

I have created a schema Relation, which has two entries, teacher and student, referring to the ObjectId in the User Schema.

var RelationSchema = new Schema({
  teacher: {
    type: Schema.Types.ObjectId,
    ref: 'User'
  },
  student: {
    type: Schema.Types.ObjectId,
    ref: 'User'
  }
});

It is trivial to query all the students that one teacher is in charge of. ---- (1)

When it comes to querying its complement set, it becomes less obvious. The first thought is to query all the users minus the results from (1).

User.find({}, function(err, users) {
    if (err) { /*handle and return */ }
    Relation.find({teacher: teacherId})
    .populate({path: 'student'})
    .exec(function(err, relations) {
       if (err) { /* handle and return */ }
       // map relations to students
       var students = _.map(relations, function(relation) {
           return relation.student;
       });
    });
    // Return the difference of users and students
    // i.e., complement = users - students
});

However, the above method might cause huge overhead.

What is the most efficient way of querying the complement set of the students that one teacher is in charge of?