Acts 13:22 gives us insight into this phrase as it also covers the narrative of King Saul’s replacement by David.
“After He had removed him, He raised up David to be their king, concerning whom He also testified and said, ‘I have found David the son of Jesse, a man after My heart, who will do all My will.’
Acts 13:22 (NASB95)
Here we see that a person after God’s own heart will do God’s will.
My father once explained to me that God has a first will and a second will and so forth. My father’s example from marriage showed God’s first will is for us not to marry, but to remain single; However, God’s second will for us is to go ahead and marry if we desire to or cannot settle for the unmarried life.
Regarding sin, there is also God’s first will and second will. God’s first will is that we avoid sin all together and not sin at all. But Romans 3:23 looms in our face reminding us of our latest failure, “for all have sinned and fall short of the glory of God.” The idea of not sinning became so foreign to the Israelites that they indulged in such willful sin that God finally had Hosea speak to them saying, “For I delight in loyalty rather than sacrifice, And in the knowledge of God rather than burnt offerings.” God’s second will regarding sin is that we repent and seek forgiveness of our sins whenever we do sin. This was the purpose of the Old Testament sacrificial system and of Christ’s advocacy of the Christian before the Father.
King David sinned. Yes, he sinned greatly. Yet he also sought God’s forgiveness. David’s response when the prophet Nathan confronted him about his sin was, “I have sinned against the Lord.” And further from Psalm 51.
Wash me thoroughly from my guilt
And cleanse me from my sin.
For I know my wrongdoings,
And my sin is constantly before me.
Against You, You only, I have sinned
And done what is evil in Your sight,
So that You are justified when You speak
And blameless when You judge.
Saul offers up a whole menu of excuses when Samuel confronted him over his sin. There is no remorse, no repentance, as seen with David. Here is the difference between the two men and here we find the key to living as a person after God’s own heart through living a life of loyalty to God and repentance of sin.
 New American Standard Bible (1 Samuel 13:14). (2020). The Lockman Foundation.
 See 1 Corinthians 7
 New American Standard Bible (Hosea 6:6). (2020). The Lockman Foundation.
 New American Standard Bible (2 Sa 12:13). (2020). The Lockman Foundation.
 New American Standard Bible (Ps 51:2–4). (2020). The Lockman Foundation.