This is likely due to the way the calculation view _sometimes_ deals with joins.
In some cases it is not possible to break up the column views involved in a join and create an execution plan that would join the underlying tables directly without having to materializing the rows.
If this cannot be done, then intermediate tables are filled with the data "so far" and the join is then performed against these temporary tables.
And these temp tables also reside in memory.
That's partly why you typically don't want to join the outcome of analytic and/or calc views.
The other part of why you should be careful here is: semantics. With tables it's pretty straight forward to tell what it actually means to join two tables in your data model.
With aggregated and processed data this might not be as straight forwards - but you have to define it, since otherwise the result of the join doesn't mean anything...