I have come to see these two verses threaded gloriously together:
"My house shall be called a house of prayer." "Do you not know that your body is a temple of the Holy Spirit within you."
If someone lives in a particular house, chances are they chose to do so. If they chose to live there they likely had reasons for choosing to do so. Therefore if the specifically desired circumstances were not present then they would likely choose another place to dwell.
God also has specifically desired circumstances for the place where He dwells. He has not said of His dwelling place it shall be called a house of beauty, nor a house of glory, nor a house of wisdom, nor a house of peace, though the place where He dwells includes all of these and much more. He has said, “My house shall be called a house of prayer.”
Consider the progression from the Old Testament to the New Testament, the transition between God dwelling in the Temple to His dwelling in His people.
"And when the priests came out of the Holy Place, a cloud filled the house of the LORD, so that the priests could not stand to minister because of the cloud, for the glory of the LORD filled the house of the LORD."
"And I will give you a new heart, and a new spirit I will put within you. And I will remove the heart of stone from your flesh and give you a heart of flesh. And I will put my Spirit within you, and cause you to walk in my statutes and be careful to obey my rules."
"And I will ask the Father, and he will give you another Helper, to be with you forever, even the Spirit of truth, whom the world cannot receive, because it neither sees him nor knows him. You know him, for he dwells with you and will be in you."