I'm actually surprised by the number of questions I receive concerning the people pane. It seems that some Outlook users are worried that the list of email, attachments etc they see on the bottom of an email message are being sent to the recipient. The Contact photo that displays on messages also generates a lot of questions and requests to disable Outlook from sending the photo.
The user is concerned that the Pane allows other users to see information, e.g. a confidential appointment or private email.
The people pane is not sharing your email and other data. You are not sending your photo with your messages.
The people pane is basically a search folder for messages and appointments in your mailbox that are to, from, or with, the selected person. In many messages, it will be either you or the sender/recipient, unless the message was sent to a group. Selecting each person will show you messages to or form that person, social networking feeds (based on their social network security settings), appointments or attachments to or from the person. You'll only see what is in your mailbox or what they have allowed on social networking sites. Others will only see what is in their mailbox or what you allow on social networking sites. You can't see in their mailbox, even if you use Exchange server.
The photo you see on messages is also not sent with the message. If the recipient has a contact for you and added a photo to it, they will see that. They could also see your Facebook or LinkedIn photo, if your security settings on either site allow it.
Occasionally you might see a photo that does not belong to the sender - in my case, email from GoDaddy displayed pictures of two sour looking men. I finally made a contact for GoDaddy and added their logo as the contact photo to have something better to look at. I've heard similar stories from others.
I think the levels of integration between services and the prevalence of services such as Facebook which do share your data is making it difficult for people to be sure what is being shared and what isn't. I've noticed on Google Plus extra information appears for personal contacts but there is clear information to disinguish this e.g. "Visible only to you".
Absolutely. It's really confusing to know who can see what.