You will not just stumble across the Begijnhof. It is a small courtyard or part that is really accessible to the houses around it. You can gain entry via a small passageway that cuts though the surrounding buildings.
There is a small church also in the courtyard, but it was not open to the public. One claim to fame of this area is that the oldest building in Amsterdam is here. Truly the architecture of Amsterdam is unique and here you can see it close up without all of the noise and bustle of the open streets.