Generally speaking, the passive voice of the verb (the subject has the action done to it) is weaker than the active voice (the subject does the action) in incorporating the reader in your story. That is crucial because as a memoir writer you are not sharing ideas but recreating life experiences to share.
"Mary baked a cake" is active. Mary, the subject of the verb baked, is doing the action of baking. On the other hand, "A cake was baked by Mary" is passive. Here Mary is not the subject. The cake, which is being baked by Mary, is the subject. The action is being done to it.
The passive voice has less impact than the active voice. The reader will experience it as an evasive attempt on your part to not "own" the action of the verb.
Dependence on the passive voice may reveal the writer's own passivity. The writer is perhaps having difficulty coming to "ownership" of the topic.
"The requirement to wear long sleeves was returned by the women" evinces less ownership than "The women revised the requirement to wear long sleeves." In the passive, the writer does not own refusing to wear long sleeves. One notes the writer seeks distance (even if unconsciously) from the action.
If you use many passive constructions, ask yourself why you do so. Would using the active voice help you come to ownership of your stories – and at times of your life?
Remember that as memoir writer you want to get your reader involved with the story. Nothing does this better than the active voice.
Good luck writing your memoir.