Abir Roy : Hello and welcome to the latest episode of Sarvadha Vartalap. Today we'll again discuss a very important topic. And just to give you a background, there have been several white papers which have been issued by various antitrust regulators, including our competition commissioners India, on the interface between AI and competition law. They have identified issues and suggested some self-audit mechanisms.

Which is what we discussed today, as to while we understand the risk from a theoretical perspective, what should the companies do practically? And to discuss this important issue, I have with me my colleague Aman. Aman, welcome to Sarvada Vartalap.

Aman Shankar : Thanks, Abir. Good to be back to discuss such an excellent and exciting topic.

Abir Roy : Excellent. So let's dive in straight away. While we understand the risk, we'll touch upon those risks identified in the white paper by the CCI in this podcast. I think I'll start with a very fundamental question as to what should be the actionable steps for the companies now and why is this issue of AI and competition law so relevant in today's context?

Aman Shankar : I think I'll be the important point here is that a year back or so we were seeing AI as just a mere technology, but right now it has transcend that boundary. is operational and it is very much integrated into the core of businesses and that's where it is also driving if we talk about digital business, it is driving right from visibility, access ranking, pricing, recommendation, defaults and interoperability and when these factors come into picture and play obviously competition law has to step in, in some way or another. Because ultimately these AI are embedded right in your product design, your data flow, your pipelines, your vendor contracts are also around these use cases. Ultimately competition law will have to step in there and that's why we are discussing this issue and even regulators have taken a cognizance of this new age technology.

So the starting point is very simple. When you talk about compliance on a broader principle level than every serious organization that is operational whether it may be a service recipient in the form of AI or maybe a developer doesn't matter, you should treat AI competition compliance as an operational discipline. Not as a one-time legal memo kind of thing, which you just tick box and add competition questions to your compliance module wherein you ask that are your product design competition law safe?

What are your procurement and mechanisms around AI? What are your vendor onboarding activities? What are your customer contracting activities? How do you launch approvals and model change logs? 

So all these things have to be taken into account when you talk about on a broader principle level speaking of AI compliance and governance.

Abir Roy : I think it's so interesting when you spoke about this, and this is something that we as practitioners and even the regulator is facing, is that when we talk about competition law till say a couple of years back, we were all comfortable with contacts which are human in nature, human interface. 

For example, whenever somebody comes out to us and said, okay, fine, we have received a DG notice, for example, the first thing that we would say, okay, fine, give your email dumps, give your minutes of the association meetings, give your contracts, give your policies, everything which is written. 

So I would say if I may bracket it for a lack of a better word, traditional - that used to be the practice as to what the on-ground human conduct and human policies and written policies look like. Now with the AI coming in, it is so dynamic in nature, like you just said, I'll just take one word like you said, ranking for example. How does the ranking in an e-commerce website happen? How does personalized pricing happen? So these are very, very interesting perspectives, which quote unquote is “machine learning”, “machine ready”, etc. So that is something that we need to look into from the AI and competitional perspective.

Aman Shankar : Yeah, and exactly why the regulators not only the CCI is I will not say worried but taking a precaution about these autonomous self-learning opaque and data rich technologies wherein it can also generate similar outcomes that we have seen in traditional markets be it about collusion, discrimination, foreclosure or exploitative pricing so to say. And even when there is let's say no traditional agreement in the sense that we have conceived it till date under the law. 

So the ultimate point is that the risk involved isn't only about the conductor also. It's also about the evidence when you talk about these kinds of technology wherein human interface is there but it's limited in all these scenarios that we are talking about. So what's the evidentiary standards, evidentiary tests that will come in. Can the businesses later explain why the algorithm behaved in a certain way? Because ultimately the liability or the obligation rests upon you. So these things will also come into play.

Abir Roy : We'll take this explanatory concept slightly later, but I think let's create a, I would say, a background as to what are the issues that the CCI looked into. The end goal of the CCI in the market study was if you have AI market-driven power and you're contributing or committing certain acts, how should those be to be compliant? So they are setting a compliance expectation. So that is the end goal from the self-audit perspective. 

From a risk perspective, highlight, and I think that is common with all the regulators. They have spoken about self-referencing, they have spoken about entry barriers in terms of data and computation, etc. They've also identified certain practices from a collusion perspective, that is, algorithms, etc. So I would say the classic cases of anti-competitive behavior have been identified in the CCI market study. And that is something which needs to be looked at when we discuss how the compliance expectation should start.

Aman Shankar : So, let me just ask this question. We have understood and broadly so from the AI market study that the CCI did or other regulators have done that what are the principles when we talk about self audit as far as AI governance is concerned. But practically speaking, it's also about how do you implement it? So in your mind, what's the first actionable step that will come into play?

Abir Roy : Think whenever, and this will apply to any new age law which is coming in I would say, even for say our privacy laws, for example, and this will come in first and foremost is data mapping. As to how data flows in your organization and with your channel partners, your vendors, like you spoke about in the introduction. So, data mapping is a critical aspect of compliance, that is point number one. 

Point number two is from data compliance or data mapping, you need to understand how it affects the market or competition. Like for example, you have all these algorithms like we mentioned, pricing signals like you mentioned, personalized algorithms that we spoke about. How does it affect say pricing access? Is there any exclusion, any interoperability concerns? So we need to see how it affects competition. 

And thirdly, comes a very important part of compliance is compliance logs. As to, I think you hit the nail on the head in the initial remark when you said about tomorrow if the regulator were to ask you as to how you came to this conclusion, you cannot reconstruct the event. You need to show from contemporaneous compliance logs as to when a new product has been introduced was the product tested? Was the stress testing done? Is there any human escalation? Because that is very, very important. That there should be a human in the loop. So that is extremely important. 

So I would say from an extreme vision perspective, without going into implementation, from a vision perspective, I would say build compliance logs. So I would say just to recap, I would say first data mapping. Second, how does it affect competition or market behavior as such. And third, I would say build compliance logs.

Aman Shankar : Okay. So essentially what you're saying is two things that I can draw is obviously build your own AI competition law inventory and second your justifications, whenever the regulator asks you shouldn't be an afterthought and and when you distill these principles it includes things like - list every system AI model AI system tool or workflow used in the organization, document it properly. For each record, let's say the owner, what is the business purpose? What are the data inputs? Output type? What is the decisional impact, because that's a very important criteria for the affected stakeholders and vendors involved because vendors will also play a very critical role here and ultimately whether it influences pricing, access, ranking, recommendation, defaults or your contractual terms. 

And secondly, what I understand is that every material AI system should have a named accountable owner, as you were mentioning, because accountability has to be fixed. It can't be left to the machines. And when you talk about accountability naturally a legal escalation route has to be internalized and has to be followed strictly within the system wherein the senior management would have the visibility of a high-risk AI system. Because a graded approach also has to be taken for your AI systems. Not everything is high-risk. Not everything is low-risk. And ultimately the stakeholders that are going to be impacted will obviously be one of the criteria for your gradation. 

And finally the document approval which comes whenever a new kind of AI model is going to be deployed or any existing model has been tweaked or logs are changed, there has to be a documented approval system for that purpose. Because ultimately what we are talking about today in anticipation is that these are the systems that can impact or influence your pricing, ranking, access, recommendations as we discussed. So when you talk about such high risk systems which can upturn competition in the market, you have to have a due diligence measure that is not only just on paper but also operationalized in conduct.

Abir Roy : I think you've articulated it perfectly. I think and I would take one word that you said in the introduction that AI is no longer a mere “technology”. So when you talk about AI compliance, it cannot be just a technology register, if I may say so. It's a compliance risk map. So you need to map the systems extremely well. And I would then go to some extent to say there are two things that we need to consider just from an earlier question that you asked - as to what is implementation. 

I think one thing that I think I should have harped upon is where do you sit in the AI value chain? Are you a developer? Are you a deployer? Are you an end user? Where do you sit? Because accordingly, the compliance would have to be tailored as to where do you sit in the value chain? That is point number one. 

Point number two is again when you mentioned accountability and systems. We all know that one of the major aspects of any AI system is how the AI system is trained. The training of data becomes extremely important. So that is another thing which is very important is that what is the data on which your AI system has been trained? Is it based on publicly available data? Or is it available, is it maybe confidential data of competitors, vendor specific data? I would say proprietary data.

So, everything also depends on how your data is being trained. In fact, companies will be well advised to have a blacklist approach. As to okay, fine, this is a blacklist approach that this is something that you should not be training your data on because that is extremely important from a data management perspective. If there is a blacklist approach, then you can say that this data should not be ingested into the system at all. So that is something I would say from a very, very top level approach that this blacklist approach should be adopted.

Aman Shankar : Abir, I think we have spoken at length about the various principled policies that have to come into play for organizations.

But you know often these principles or compliance policies whether it be competition or any other law often remain abstract? Especially when you talk about a technology that is dynamic and rapidly evolving. But now I'm taking a step further. How do you think that any organization who is dealing in these kinds of technologies can make this operational and not just a mere abstract legal theory?

Abir Roy : I think this is a fair question because this is a risk that I also see from a traditional competition law perspective or any compliance that it sits well on paper, but how do you implement that in practice? Because that is extremely critical, wherein just having a policy on paper would not do. So I would say have all these risks that we have identified, I think convert them into a trigger point.

When I say trigger point would be okay, fine, if your pricing that an AI company suggests to you is below cost, that should be a trigger as to why is it happening. We spoke about ranking. If there's any change in the ranking, then why is it happening? So these can be the trigger points that can be developed into the system when the product is designed to see. 

Because one thing which is there is we hear about self-learning algorithms, etc. But they cannot be audited only once and forgotten. Their monitoring should continue over time. There should be clear safeguards which have to be provided, and clear product triggers, I would say. Like I said, below cost pricing, certain ranking shifts, etc. as the case may be. And when these triggers are hit, we can categorize them into high, medium, and low. And depending on that, they should be translated into a human review.

That is extremely important. And the human review should be in such a manner that they should have a kill switch. When I say kill switch, I mean okay, fine. If there is an issue, they should immediately stop it from happening. At the end of the day, like I said, this always ties back to the point of compliance logs that we mentioned. and that is something which is critical from an implementation perspective.

Thirdly, I would say just to complete this flow of loop is I would like to tie up the point of training data also. And we have seen some cases internationally, and which the CCI also relied on in the market study also, is about third-party AI tools. So when you have an agreement on an arrangement with a third-party AI tool, you should say that okay, fine, what kind of data should your system be trained upon when you are suggesting an output or a recommendation to the company concerned. 

So those things may also need to be built in from an AI systems perspective.

Aman Shankar : I think what you are suggesting is if I can borrow from let's say the new data prediction law is A, compliance by design and B, there has to be auto triggers. And when every incident is recorded that automatically makes your entire compliance program also auditable because ultimately you also have to do the impact assessment at the end of the day if you are dealing with high graded risk AI systems.

Abir Roy : Yeah, exactly. In a sense, like you said, I think you use the term correctly, compliance by design. And these compliance mechanisms should be embedded in the system design itself. I think I come back to the key question or the larger question that I asked initially is that you should not have a policy written somewhere. It should be actually implementable and explainable. I think explainability is the key, which I think most of the white papers including but not limited to CCI have mentioned about that you need to explain your conduct which may be which may be a result of an AI design.

So the question is not whether the organization concerned had a policy somewhere. The question is whether the policy, which is a written policy, triggered a human review at the right time and the right place, and the right action was taken. Second, I think the most critical part is what the market study also said, and all various white papers written by various international authorities, is the concept of explainability. Can you explain the conduct, which is the major point.

Aman Shankar : To put it very simply, an organization should be able to explain why the system produces a certain kind of output, especially when you're dealing with such kinds of AI systems and where transparency becomes a key. And this is not by disclosing your source code or trade secret, but by simply doing the exercise of maintaining records of key parameters as to what are the inputs, what are the model version, how does the model decide on different kind of inputs that are given, what are the decisional rules that come into play, if there is any human override that is possible and ultimately the business rationale. And I would like to specifically emphasize on the point of transparency because that has become a key challenge when we talk about this new age technology or even in the digital market cases that we have seen in the past the transparency often becomes a key wherein the businesses often argue on the defense side that it would lead me to disclose my trade secrets or IP whereas the other side simply demand a fair competition of principle of transparency. 

So transparency here as I said is not about disclosing your source code or your IP and it's also not about disclosing about your commercially sensitive model architecture. It means being able to explain the purpose of the system and the broad parameters, what are the factors that influence your decision, any human override points that are possible, the risk controls that are integrated and embedded, the reason a material output was produced and if two comparable customers received different prices or two, let's say comparable business receive different kinds of output on the same nature of input, then there has to be a the organization must be able to show that what are the yardsticks, rational note of the objective basis with over which those inputs or outputs were done by the AI systems in place. 

So from a pure compliance perspective a one-pager model fact sheet on these principles is a very practical starting point. So to say you have to run, you know, simulations for collision like price matchmaking or stress test for extreme demand or supply changes, shadow competitor tests for self-referencing and bias tests for differential treatment between similarly placed users. And keep those test results as a document when you talk about compliance and should not be a mere conclusion, but it should be a continuous exercise that goes into place. And then only when you document entirely everything, you have the rationals, you have the objective basis for everything, explainability as a defense comes into picture and becomes a very solid ground for you as a business.

Abir Roy : I think you'll hit the nail on the head when you use the term objective, because the CCI in the recent past, borrowing from international principles, have actually adopted the principle of objective justification for any conduct. So when you can explain a conduct, it's no longer a black box. I think one of the concerns with any algorithm or any system is the fact that it should not be opaque in nature or a black box. Like you said, it should need to explain itself properly without obviously disclosing your IP, etc., as a case, maybe. So proper documentation becomes extremely important, and proportionality is the key. 

So that I would say is very, very essential. So let me just recap. I just take this opportunity to recap whatever we have discussed into say two to three buckets.

First, Aman, like you said, first would be the identity layer of any system. Who is accountable? Which is the human or the human team, which is accountable. That I would say is step number one. 

Step number two is I would say the delegation layer. If you build a system, what is the system authorized to do? And most importantly, what not to do? Like I said, you should not be trained on say a blacklist item. That I would say step number two. 

I would say step number three would be the execution layer, which flows from the delegation layer. 

Fourth will be the evidence layer as to if somebody were to ask you a question. As a regulator, you need to have evidence to explain yourself. That is a concept of compliance logs. 

Fifth would be again coming back to the human in the loop, there should be control over the system.

And sixth and most important would be the compliance layer. We have not only AI, we have so many laws in place which deal with platform economy and AI, etc. You should have the systems. You should have a compliance layer to show that it is compliant with various laws.

Aman Shankar : I think Abir, you know, I'll also try to bring in a very different point here, which is very important when you talk about compliance by laws. I already spoke about how we can borrow the principles from privacy law compliance by design and self audit and impact assessment. And this is where competition law, data protection, consumer protection, platform regulation, contract law all converge together when you talk about EI regulation as such or AI governance as a principal based outcome. A default that bundles consent for multiple AI use cases may raise a privacy concern, but it may at the same time also be the consumer law concern or a competition law concern wherein these factors may indicate, let's say, a market power. We have seen that in the past. Your API integrations or restrictions may also be seen as a ground for innovative practices, but at the same time it may become a competition law issue or it may result in a contractual law dispute. 

So when you talk about convergence of these laws, what the future beholds is not just seeing laws in silos, but actual convergence of all the laws when you talk about AI regulation and litigation, it will be how different regulatory lenses evaluate the same kind of design choices and one has to be conscious of various laws interplaying with each other. Because we have also seen a, this has been always but has become more materialized that every law at least in India uses this particular provision where it says that this law is in addition to other laws. So that's how the convergence comes to play.

Abir Roy : Yeah. In fact, yeah. In fact, it's a very interesting point. I'll just have a deep thought here when you speak about design choices. And you specifically mentioned consumer laws. You all must be seeing all these notices which are coming on dark patterns, for example. That is nothing but platform design and product design. So that is also there's an AI perspective also, because when you talk about drip pricing, or any other kind of practices which are there. One of the defenses is it’s all AI generated or algorithm generated. Now you cannot take the defense ideally because it's not that it's a third-party algorithm on which you don't have control. It's a human review, it becomes extremely important. 

So I would say you're right. When you talk about convergence of laws, there's so many laws which converge on the similar practice, which is it becomes extremely important when you obviously adorn the advisory hat.

So now I think we have now touched on it slightly, but I think it may be fair to again delve slightly deeper as to what are the practical risks when you talk about AI and how does it affect say a competition or market behavior? What you would say are the top three headline risks which we are talking about?

Aman Shankar : I think to start with the five top bucket list for me first will obviously be the concentration in the AI value chain because this technology by its nature is resource heavy, investment heavy, having various layers and all being concentrated. So the first is concentration in the AI value chain. And when you talk about this companies must document their position at each layer of the AI stack. There they should also review exclusivity in data and supply contracts because that is the main input when you talk about AI. They should define the data access policies that have maintained, bias-testing records and keep home human oversight ultimately meaningful over all of this. Because we have spoken at length about that accountability has to be fixed so human oversight has to be always there. 

Number two will be for me the ecosystem lock-in and switching costs. Now the practical steps here are that data portability has to be there. There has to be export testing contractual clauses that allow users to multi home and there has to be a comparable API access where commercially and legally appropriate. It may be obvious that business has to make sense. But API access has to be at commercially and legally appropriate standards. 

Number three will be collaborations and partnerships now any AI related deal should go through a competition screen checklist and the questions that should be asked is does that arrangement give a unique access to data, compute, chip, models or distribution. Could rivals be foreclosed and are information firewalls clearly defined in the agreements that you are going to enter into. So this is number three for me. 

Number four would be self-preferencing. We have spoken at length about it, but the actionable point in my mind isn't just to stop every integrated product strategy. It's to document the objective criteria because that's our defensible ground in competition law. You have to document the objective criteria that if you are going to undertake a certain kind of measure or a step, how is it going to A, further your business, B, result in consumer benefit and C, it is not going to have any adverse impact or effect on let's say ranking, axis, defaults, recommendations or your entire algorithmic outputs that are going to be in place. 

And the last but not the least the point will be and we have spoken at length about it is pricing coordination. Don't train your AI models on sensitive pricing information of your competitors because ultimately that becomes a challenge and that's how we have seen cartel cases come into picture. Always vet the third party vendors that are being onboarded. I already spoke about that there has to be information firewalls, but especially for sensitive data that includes pricing data, you have to have information firewalls. Prohibit the use of competitive sensitive data as we already spoke about. Train your sales and support team not to imply that tool that gives access to rival strategies and keep humans responsible for significant pricing decisions. This is I think the ultimate undertone of the entire actionable point, which I have said the ultimate undertone is obviously that humans always have to be in the loop.

Abir Roy : It's so interesting that you mentioned all of this. And I have one more which I would like to speak about. But before that, I think one of the things which you mentioned was about self-referencing. And that is actually which has caught the attention of all the regulators worldwide is to where do you draw the line between interoperable products, which may be a result of efficiency, and where does self-referencing as a concept become an abusive conduct. 

And that as it is, I would say the issue was extremely complex, which has become more complicated because of AI. So that is something to be watched out for. And explainability like you said, cannot harp it enough, is the key. I would just add one more which you mentioned on pricing algorithms and I'll stretch that to say price discrimination or any kind of discrimination as a result of AI. 

Now, discrimination obviously there is no justification becomes an issue. So that is something that needs to be looked at. One of the reasons why objective justification was seen as a defense in the BookMyShow case was because it was proportionate and it was objective. So build objective justification in case of discrimination. If you are discriminating.

And the last would be, I think, is again the black box nature of the algorithms. Nobody is asking, like Aman, you mentioned, that the fix is not a full disclosure. And NCLAT also in the Google case said that there's no need to fully disclose the APIs because at the end of the day, APIs are proprietary systems of Google. So the fix cannot be a full disclosure because there's an IP angle to it. However, having said that, you need to have those compliance logs based on what we discussed, have properly articulated audit trails and clear points of human review is what I would say.

Aman Shankar : So Abir, we have discussed at length about the various formulations, strategies, modus operandi that can be brought into picture as well as AI governance and compliance is concerned. Now, as a recap and as a forward-looking measure for businesses who are maybe hearing this podcast or otherwise also, list the actionable points as to how to make all these principles that we have discussed at length operational in an organization.

Abir Roy : Hmm. So let me just try to bucket it again, like we discussed. First, which would again, have a team, have a fully functional, cross-functional designated AI team. That is very, very important so that you know that this is the team in charge. 

Second would be to map the AI systems, including data mapping, like we mentioned. 

Third would be conduct risk assessment, convert this risk into product triggers when you talk about risk assessment.

Fourth would be audit. And this is in line with the earlier point that we mentioned. Convert them into product triggers. So I would say have this risk and audit. 

Then would be whenever you are coming up with a new product, stress test them so that there is a proper escalation matrix and identification of the risks which are there. So I think we need to name a proper accountable AI system, like I said, from the very beginning.

Create an AI inventory risk map. Third would be to maintain a prohibited live blacklist, that such kind of data will not be injected into the system either on its own motion or to a third party AI vendor so that the data is not injected on, like you said, confidential data of any competitor. So that is very, very important.

And obviously, we need to run all these stress tests, like we mentioned. One point obviously, then you need to review your customer and vendor contracts, if there is an exclusivity, etc., why it is there, then train your models accordingly. Have like we mentioned that it cannot be a one time compliance memo. It should be a live document so that it has to be impressed upon from the cross functional AI team. That this has to be a repeat and a continuous exercise of compliance. 

And last but not the least, is again, we'll harp on it again, is explainability. Let me just make it slightly more casual when I say that we all watched the famous line of Denzel where he said, explain this to me as if I'm a five-year-old. I think that is the key. That any AI system, when asked by a regulator, you need to explain the concept in such a simple tone and language that this is the compliance that we are doing. This is the AI system that is meant to do. This is the result because of the compliance. So then you know that the compliance is not a memo in a system which is just downloaded somewhere, it is actually implemented in it.

Aman Shankar : Very interesting because you know just to summarize and sum it up since we are towards the end of this podcast and say the key principle that has been echoed in your actual points also is that continuity.

Continuity is the key and because you're talking about a technology that is so dynamic and rapidly evolving, the use cases are changing the models, the system courts the logs everything is going to change. So self audit even if you talk about self audit has to be a continuous process that has to keep up with the change in technology. The message is I think very clear that AI competition compliance is less about abstract principles, but is more about evidence, systems and repeatable processes. 

And businesses should I think ask themselves a very practical question : if the CCI asks tomorrow why your AI behaved a certain way, why your pricing decision or a recommendation was taken in a certain way, why the output was given in a certain way for two different customers? So what you should come up with and what you should be able to explain is that there are contemporaneous records recording every kind of decision-making. If the answer is yes, then definitely you’re proceeding ahead in the right direction. If the answer is no, then there is a reason to rethink your entire AI governance policy. And I think the message that has been quoted by the competition regulator also with the AI market study that has come into place is that the regulator is not taking a reactive approach to these kinds of new age technology or the conducts that we are talking about. It's more on the side of a proactive approach wherein businesses are already informed that what are the standards and yardsticks on which the regulator requires you to behave in the market and what are the defenses that you can come into play, what are the objective justifications that come into play whenever the system goes haywire in the future or any let's say you are involved in any kind of litigation. 

So I think with this parting thought we come to the end of this Vartalap. And I would say that let us all stay hopeful about this new age technology and what kind of innovative pro-competitive effects it is going to have in the market, but till then thank you for watching this Vartalap and stay tuned for our next episode.