[Oliver Cowdery] James J. Strang began to "gather" his followers to a new colony on Beaver Island in MI. By late 1850 his old headquarters at Voree, Walworth, WI was largely deserted. Some apostates from his group, led by Aaron Smith, remained in Walworth Co. after Strang's departure to MI. Lyman Cowdery may have had friends among Smith's followers.
[source: Broadhurst, Dale R., Oliver Cowdery Chronology, http://olivercowdery.com/history/Cdychrn1.htm]