Introducing Avatar: WorldExplorer in Pyjam application on mobile app stores: This is a cross-platform application for remotely hiring freelancers. You can hire a freelancer from anywhere in the world by selecting them on a map and controlling them with a joystick. The freelancer will broadcast their surroundings through their camera and follow the client’s instructions.
To ensure that the application, Avatar: WorldExplorer in Pyjam, works on iOS, Android, and web browsers, we decided to use the Ionic framework. This framework is capable of packaging a ready-made web application into the native part, allowing us to reduce staff costs and accelerate development time. Essentially, we write in Angular, build static files, and then these files are packaged into a container.
It works similarly to PWA applications, but the frontend is already inside the application and essentially opens the address http://localhost/, and then the frontend connects to the backend and everything works as needed. However, we encountered a problem when we realized that we needed to make an incoming call to the performer. When our app is minimized, it essentially stops working. And then, what if the phone’s screen is locked? We needed a solution for the performer to instantly know that someone wants to control them.
We looked at how this is implemented in Telegram (their client-side source code is open) and how it is done in VoIP programs and realized that the best solution for us would be to somehow use the system call interface. At the moment, we have managed to do this for iOS and are almost finished for Android, but the scheme is similar.
Our actions in iOS:
As it turns out, to make the system call interface ring, you need to send a special push notification with a VoIP tag, which will invoke the system call interface, even if the screen is locked, as if calling through the GSM network. When you press the “Answer” button, our application will open and users will enter into a video call. It was decided to use PushKit and CallKit. We couldn’t use a ready-made plugin that works through Firebase because it cannot send such a push. A mandatory requirement for using VoIP push is the need to display the call interface through CallKit; otherwise, Apple might block VoIP push notifications for some time. It turns out that we needed to write code in the native part of the application.
It was also necessary to obtain a special certificate for VoIP from Apple. And to configure XCode, there are many articles written about this, so I will not go into detail here.
Here is part of the code where we implemented the call using CallKit.
``` func pushRegistry(_ registry: PKPushRegistry,
didReceiveIncomingPushWith payload: PKPushPayload,
for type: PKPushType,
completion: @escaping () -> Void) {
if type == .voIP {
let callUUID = UUID()
if let orderId = payload.dictionaryPayload["avatarOrderId"] as? Int,
let name = payload.dictionaryPayload["name"] as? String,
let surname = payload.dictionaryPayload["surname"] as? String {
let callConfigObject: CXProviderConfiguration
if #available(iOS 14.0, *) {
callConfigObject = CXProviderConfiguration()
} else {
callConfigObject = CXProviderConfiguration(localizedName: "\(name) \(surname)")
}
callConfigObject.supportsVideo = true
callConfigObject.includesCallsInRecents = false
// Create an object to give update about call-related events
let callReport = CXCallUpdate()
callReport.remoteHandle = CXHandle(type: .generic, value: "\(name) \(surname)")
// Enable video call
callReport.hasVideo = true
// Create an object to give update about incoming calls
let callProvider = CXProvider(configuration: callConfigObject)
callProvider.setDelegate(self, queue: nil)
callProvider.reportNewIncomingCall(with: callUUID, update: callReport, completion: { [weak self] error in
defer { completion() }
if error == nil {
self?.cacheCallAnsweredFor(callUUID, orderId)
}
})
} else {
let callConfigObject: CXProviderConfiguration
if #available(iOS 14.0, *) {
callConfigObject = CXProviderConfiguration()
} else {
callConfigObject = CXProviderConfiguration(localizedName: "Failure Failure")
}
callConfigObject.supportsVideo = true
callConfigObject.includesCallsInRecents = false
// Create an object to give update about call-related events
let callReport = CXCallUpdate()
callReport.remoteHandle = CXHandle(type: .generic, value: "Failure Failure")
// Enable video call
callReport.hasVideo = true
// Create an object to give update about incoming calls
let callProvider = CXProvider(configuration: callConfigObject)
callProvider.setDelegate(self, queue: nil)
callProvider.reportNewIncomingCall(with: callUUID, update: callReport, completion: { [weak self] error in
defer { completion() }
if error == nil {
self?.cacheCallAnsweredFor(callUUID, -1)
}
})
}
} else {
completion()
}
}
} ```
In this code, we receive a push notification, and if it is of the VoIP type, we start configuring CallKit. If there is an error in decoding the payload, we also display the call interface, but with an emergency caller name “Failure Failure.” This is intentionally done to avoid interruptions in the PushKit + CallKit call chain. If there is an error in decoding the payload, we also display the call interface but with an emergency caller name “Failure Failure.” This is intentionally done to avoid interruptions in the PushKit + CallKit call chain.
When the push notification is pressed, the application is launched, creating an AppDelegate along with all the infrastructure in the form of plugins. Upon creating an instance of the plugin, iOS registers that a PushRegistry has been created and passes a reference to the class implementing the PKPushRegistryDelegate protocol in the delegate field, the method pushRegistry…didReceiveIncomingPushWith of which intercepts the VoIP push.
AppDelegate.swift
```
extension AppDelegate {
static var shared: AppDelegate {
return UIApplication.shared.delegate as! AppDelegate
}
}
```
```
import Foundation
import Capacitor
import PushKit
import CallKit
@objc(VoipPlugin)
class VoipPlugin: CAPPlugin {
var voipRegistry: PKPushRegistry?
var invalidateToken: Bool = false
var avatarOrderIds: [UUID: Int] = [:]
@objc func register(_ call: CAPPluginCall) {
registerForVoIPPushes()
call.resolve()
}
public override func addEventListener(_ eventName: String, listener: CAPPluginCall) {
super.addEventListener(eventName, listener: listener)
switch eventName {
case "voipPushRegistry":
if let pushToken = voipRegistry?.pushToken(for: .voIP)?.map({ String(format: "%02.2hhx", $0) }).joined() {
notifyListeners("voipPushRegistry", data: ["token": pushToken])
}
case "invalidateToken" where invalidateToken:
notifyListeners("invalidateToken", data: nil)
default:
break
}
}
func pushTokenUpdate(_ token: String) {
if hasListeners("handleOrder") {
notifyListeners("voipPushRegistry", data: ["token": token])
}
}
func answered(_ uuid: UUID) {
if hasListeners("handleOrder") {
if let avatarOrderId = avatarOrderIds[uuid] {
notifyListeners("handleOrder", data: ["id": avatarOrderId])
ended(uuid)
}
}
}
func ended(_ uuid: UUID) {
avatarOrderIds[uuid] = nil
}
func cacheCallAnsweredFor(_ uuid: UUID, _ orderIds: Int) {
avatarOrderIds[uuid] = orderIds
}
}
```Here, the events that occur during the pressing of the “Answer” or “Decline” buttons are described.
Why I decided to write this:
Because we tried many plugins like Cordova-Call and others, but could not find a solution. A lot of time was spent to implement something of our own. Yes, there is not much code here, but a lot of time was spent on research.
Well, if you are interested in more details about our program, here is the description:
- Avatar: WorldExplorer in Pyjam
- Human Remote Control.
- Unveiling the Future of Travel: Avatar Program for Virtual Tourism.
In an era where technology continuously reshapes our daily lives, the concept of travel has transcended traditional boundaries, introducing us to the innovative realm of virtual tourism. The Avatar Program for Virtual Tourism emerges as a groundbreaking solution, offering an immersive exploration experience from the comfort of one’s home. This program leverages the power of digital avatars, enabling users to traverse global destinations through the eyes, ears, and movements of a remote-controlled human avatar.
The Dawn of Remote Exploration
Imagine touring the cobblestone streets of Rome, exploring the vibrant markets of Marrakech, or wandering through the serene gardens of Kyoto, all without the need to pack a suitcase. The Avatar Program for Virtual Tourism makes this possible. At its core, this program is designed to democratize travel, eliminating the barriers of physical, financial, and logistical constraints. It’s a digital bridge connecting curious minds to distant lands through real-time, interactive experiences.
How It Works
The Avatar Program operates on a simple yet sophisticated premise. Users select their desired destination from an expansive global map, showcasing available avatars in various locations. Once booked, users can direct these avatars via phone or computer, guiding them through streets, landmarks, and attractions. Through high-definition video streaming, users witness a live, unedited view of the world through the avatar’s perspective, complete with the ability to interact with the environment, communicate with locals, and even purchase souvenirs.
Beyond Sightseeing: A Multifaceted Experience
Virtual tourism through avatars offers more than just sightseeing. It’s an educational journey, a cultural immersion without language barriers, and a tool for detailed exploration of potential real-world travel destinations. Additionally, it serves unique use cases such as remote property viewings for potential buyers or renters and personalized guidance for those planning future trips.
The Technology Behind the Magic
The backbone of the Avatar Program is a sophisticated blend of real-time communication technology, live-streaming video, and user interface design that prioritizes ease of use and immersive experience. Security and privacy are paramount, ensuring a safe and respectful interaction between avatars and the communities they navigate.
The Future Is Here
As we look forward, the possibilities of virtual tourism and the Avatar Program are boundless. With advancements in virtual reality (VR) and augmented reality (AR), the program’s future iterations promise even more immersive and interactive experiences. This technology not only offers a new way to explore the world but also holds the potential to revolutionize education, real estate, and remote assistance services.
Conclusion
The Avatar Program for Virtual Tourism represents a significant leap forward in how we think about travel, exploration, and cultural exchange. It embodies the spirit of adventure and discovery, untethered by physical limitations. As this technology evolves, it will continue to break new ground, offering unprecedented ways to connect with the world around us. In doing so, it reminds us that even in a digital age, the human desire for exploration and connection remains as strong as ever.
Virtual tourism is not just a glimpse into the future of travel; it’s a vibrant, expanding reality, making the world more accessible and interconnected. The Avatar Program is leading this charge, inviting us all to embark on journeys we never thought possible.
